RFC : xdg_surface_present() look-and-feel and implementation

[Pekka Paalanen]

On Sun, 3 Aug 2014 09:23:06 -0400
"Jasper St. Pierre" <jstpierre at mecheye.net> wrote:

> On Sun, Aug 3, 2014 at 8:43 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> 
> > On Sun, 3 Aug 2014 07:30:25 -0400
> > "Jasper St. Pierre" <jstpierre at mecheye.net> wrote:
> >
> > > On Sun, Aug 3, 2014 at 6:27 AM, Pekka Paalanen <ppaalanen at gmail.com>
> > wrote:
> > >
> > > > On Fri, 1 Aug 2014 17:08:14 +0200
> > > > Manuel Bachmann <manuel.bachmann at open.eurogiciel.org> wrote:
> > > >
> > > > > Hello everybody,
> > > > >
> > > > > I just updated the repo today (
> > > > >
> > > >
> > https://github.com/Tarnyko/weston-xdg_surface_present/commit/0aca29d4b6dbe10d5237aaf5f35f72d25db3ac30
> > > > ).
> > > > > The "xdg_surface_present()" request not accepts a timestamp (uint32_t
> > > > type)
> > > > > as an additional parameter.
> > > > >
> > > > > If different of 0, and it is the first time the surface should be
> > shown,
> > > > > the shell will check if a significant amount of time passed between
> > this
> > > > > timestamp and the actual present() request, and it it did, will show
> > a
> > > > > notification instead of directly mapping the surface.
> > > > >
> > > > > You can see a demo here (1st case immediate, 2nd case delayed) :
> > > > > https://www.youtube.com/watch?v=IDa2W_xMg10
> > > > >
> > > > > The implementation is still pretty naive, but I will improve it with
> > the
> > > > > following considerations :
> > > > > - if any of the application surfaces focused, or not ;
> > > > > - are we on the same workspace ;
> > > > > - etc.
> > > > >
> > > > > Interested in your feedback on the protocol definition especially.
> > > >
> > > > Like has already been said about the request name, I agree with the
> > > > following points:
> > > > - "raise" does not describe what it does
> > > > - "activate" does not describe what it does, activate is already
> > > >   related to decorations state etc. whether the window is
> > > >   "currently in focus" or activated, I believe.
> > > >
> > > > And I add one:
> > > > - "present" is confusing, because presenting means hitting the
> > > >   screen, and the term is heavily used in the future Presentation
> > > >   extension, which is on a lower level than shell.
> > > >
> > > > I think "attention" or some variation of it is perfect. A client or
> > > > window is requesting attention... that describes exactly what it is
> > > > about.
> > > >
> > >
> > > I'm a bit concerned about "attention", because toolkit authors usually
> > have
> > > both a "present this window" and "mark this window as needing attention"
> > > API. Under X11, one maps to _NET_WM_ACTIVATE and the other maps to the
> > > _NET_WM_STATE_DEMANDS_ATTENTION flag. This request is the former, not the
> > > latter. If somebody IMs me, the taskbar button might blink, but it should
> > > never immediately raise and show the window.
> > >
> > > There's also on guarantee that this request will do focus-stealing or pay
> > > attention to the serial at all. For users that turn the behavior off, the
> > > window will simply pop up in front of them, and if app developers start
> > > using this as a non-obtrusive way of trying to gain attention, they will
> > be
> > > disappointed.
> > >
> > > I don't want to bikeshed on the list though. I'll continue to brainstorm.
> >
> > Right, ACTIVATE vs. DEMANDS_ATTENTION. I assume these are two
> > different things in X11, as the client can simply force its way.
> >
> 
> _NET_WM_ACTIVATE in X11 says "please show this window to the user". It's
> used for instance if I have an IM open with a person, and then I click
> their name in the buddy list again. It will switch to that window. It's
> used when clicking on a link and having the web browser open a new tab in
> its existing window. Or when opening files in a new tab in gedit. There is
> an event timestamp passed along, and the window-manager will switch to it
> if it's new enough. There is no guarantee the window will be immediately
> raised and available. The window manager *might* decide to downgrade the
> request to saying the window wants attention. This is what mutter does if
> the event timestamp is too old. [0] Imagine I click on a link, Firefox
> takes a while to start up, and then I keep chatting with my friend. Instead
> of having Firefox steal the focus, the request is downgraded to a blinking
> taskbar icon or a notification, so my keystrokes now go to my friend
> instead of being interrupted mid-type.
> 
> _NET_WM_STATE_DEMANDS_ATTENTION in X11 is a hint to say "the window wants
> some attention. Feel free to show a notification, or blink the taskbar
> icon". It should never switch to the window.
> 
> The proposed xdg_surface_present() extension is equivalent to
> _NET_WM_ACTIVATE. I am fine with calling it xdg_surface_activate().
> 
> https://git.gnome.org/browse/mutter/tree/src/core/window.c#n3389

Thank you for explaining the existing semantics.

I'm still did completely understand, when you would want to give a
"needs attention" notification, but still guarantee this will not
switch to the window. I would think that not switching to the
window would be the majority of the cases, and the switching would
happen only if the client manages to pass exactly the right serial,
which would mean that the need for attention was created by a
direct user action right now.

OTOH, I see _NET_WM_ACTIVATE also as something that the compositor
easily downgrade to just a notification, like you say. Especially
with the firefox example, I would see serials as a superior way to
timestamps, as then you don't have to guess how long the user might
be willing to wait for the browser until getting bored.

I suppose there is still some reason to keep the two requests
separate.

> "Present this window" OTOH just means "map this window", and I do
> > not see an obvious reason why we would need that in the
> > Wayland protocol, as we can do the same with the first
> > wl_surface.commit that carries a wl_buffer. Sorry I haven't read
> > everything, maybe you already explained somewhere why we need a
> > "map/show" request? Do we have a way to hide, too?
> >
> > Note, that I imagine this is different to ACTIVATE, since in my
> > mind activating means getting some input focus and maybe even
> > raising to top. Quite likely I am just confused here on what you
> > mean, I'm playing based on what the names sound like.
> >
> > On Wayland, we cannot have "activate me now" request, we can only
> > have "I would like to be activated because...", and OTOH we have
> > the concept of demanding attention.
> >
> > What is the difference between demanding attention and wishing to
> > be activated? I didn't see a difference, and so conflated the
> > concepts.
> >
> > And which one was this xdg_surface_present supposed to be?
> >
> > Those two questions are my confusion here.
> >
> > If it is about ACTIVATE, call it something like "activate" then.
> >
> > > > >    <request name="present">
> > > > >      <description summary="map the surface in the current context">
> > > > >        Calling this request on a newly-created shell surface
> > > > >        is mandatory to map it to the current graphical context.
> > > > >
> > > > >        If the request is called more than once, the shell will
> > > > >        send interactive GUI notifications, so the user can bring
> > > > > the surface back easily.
> > > > >      </description>
> > > > >      <arg name="serial" type="uint" summary="serial for advanced
> > focus
> > > > management, can be 0"/>
> > > > >    </request>
> > > >
> > > > Judging from the discussion so far, I don't think should have
> > > > anything to do with the client mapping the surface. The compositor
> > > > can and should do focus stealing prevention always, and whether you
> > > > send one more request or not is irrelevant, also for mapping. Isn't
> > > > the mapping of a top-level window an implied request for attention?
> > > >
> > >
> > > We currently don't have a serial or timestamp in get_xdg_surface to do
> > > focus-prevention stealing on, and I figured that that calling present
> > made
> > > sense, since then we don't have the "map on first commit semantics".
> > > Instead, now we have a request for "show the window to the user".
> >
> > > > You didn't document what the serial really is, where do you get
> > > > one, how it is used, or what it causes, and what special meaning
> > > > does 0 have. Note, that we explicitly define serial to not be a
> > > > timestamp in Wayland, they are two completely different concepts.
> > > >
> > > > Also, considering that a serial is not cross-client thing, what use
> > > > cases would the serial here have? Could this request be used by a
> > > > client to activate a top-level window B as a response to a user
> > > > action in its window A? Would that be in or out of scope for this
> > > > request?
> > > >
> > > > In the case of App1 launching App2 and using the launch-token
> > > > protocol I sketched in the other email, App2's top-level window
> > > > will already be implictly associated with App1 launching it. The
> > > > question there is, do we need an explicit association? Like, if you
> > > > can get a new serial from the launch-token in App2, you could
> > > > "activate" already existing windows (the Kate use case) by passing
> > > > that serial with attention request. Or activate several top-level
> > > > windows, or just one top-level window that yet was not the
> > > > first one created by App2.
> > > >
> > > > Anyway, these are just my ideas, and I don't know how much they
> > > > make sense, because I don't know how these things work currently in
> > > > X11. I just got the funny idea that launch-notification would be
> > > > related here.
> > > >
> > >
> > > On X11, we have startup-notification, which is basically a hash map done
> > as
> > > a sequence of add/remove broadcast ClientMessages. When you launch an
> > app,
> > > you build a "startup notification" structure containing the thing you're
> > > launching and the event timestamp it was launched with, and send it as a
> > > ClientMessage. You then send the ID of that structure to the app, and
> > when
> > > the app finally launches, the app broadcasts that it had launched, which
> > > removes the app from everybody's tracking.
> > >
> > > The important thing is actually the middle one: telling the app what ID
> > it
> > > has. This is done with the DESKTOP_STARTUP_ID envvar.
> > >
> > > In places where startup-notification is too complicated, we have a
> > somewhat
> > > silly ad-hoc convention that's not documented. You can set
> > > DESKTOP_STARTUP_ID to "_TIME123456", and that gives the app the event
> > > timestamp it was launched with, which it can use for _NET_WM_ACTIVATE and
> > > some other things.
> > >
> > > We could reuse this same scheme for Wayland. Put an event serial or a
> > event
> > > timestamp in DESKTOP_STARTUP_ID so it knows what to pass to the initial
> > > present request.
> >
> > Very good, except I do not believe a timestamp is a good one (easy
> > to fake, does not identify anything) nor a serial (does not
> > identify anything, is perhaps not a global concept), so you need to
> > ask the server to explicitly create a cookie for you to give to
> > the app being launched. And if the "present" request uses a serial,
> > the cookie needs to be converted to serial first, or maybe rather
> > another "present" request that directly uses the cookie.
> >
> 
> I had two (admittedly silly) issues with serials that made me choose
> timestamps instead of serials.
> 
> The first is that I didn't realize that they were on input events, because
> I looked at wl_pointer.motion. I suppose we'll be in trouble if we ever
> want mouse-gesture-based window opening, but for now, I think we're fine.

Indeed, every button, key press, and maybe also touch down event
delivers a new serial, pointer motion is the only one that does
not. We are ok unless someone wants to have mouse gesture activated
without any buttons or keys pressed at any point in time.

Even wl_pointer.enter carries a serial, which means that e.g. with
focus-follow-pointer case gets a bumped serial. That would allow
focus stealing prevention to help me, when I click "send" in my
email client, move on to typing in irc in a different window, and
then fraction of a second later the email client pops up a dialog
telling it is actually sending the email... with a button to
cancel, activatable through keyboard...

> The second is that "0" is a valid value for a serial, and sometimes you're
> in a place where you want to present a window and don't have any serial to
> pass back. Initial map where you don't have a launch serial (terminal
> launching is the obvious case) is a simple example. You can imagine other
> cases, though. A browser might do this:
> 
>     setTimeout(function() {
>         window.open("http://google.com/");
>     }, 5000);
> 
> Which says to the browser to open a new window after 5 seconds. We have no
> serial to pass here. Timestamps of 0 are decidedly less valid, so there's a
> good sentinel value there.
> 
> Suggestions welcome. Both having two requests and a boolean "serial_valid"
> flag were both ugly to me. Maybe we should add maybe types to the protocol?
> :)

Yes, the problem of "I don't have a serial"... I wonder if we could
retro-fit some unlikely value as a "no serial" value, like
0xffffffff. That would be my immediate proposition. Would that work?

I see that equally valid as deciding that timestamp value 0 means
"none". The timestamp wraps around too, and likely much more often
than a serial.


Thanks,
pq