EFL/Wayland and xdg-shell

[Daniel Stone]

Hi,

On 15 April 2015 at 23:51, Carsten Haitzler <raster at rasterman.com> wrote:
> On Wed, 15 Apr 2015 20:29:32 +0100 Daniel Stone <daniel at fooishbar.org> said:
>> On 15 April 2015 at 02:39, Carsten Haitzler <raster at rasterman.com> wrote:
>> > not esoteric - an actual request from people making products.
>>
>> The reason I took that as 'esoteric' was that I assumed it was about
>> free window rotation inside Weston: a feature which is absolutely
>> pointless but as a proof-of-concept for a hidden global co-ordinate
>> space. Makes a lot more sense for whole-display rotation. More below.
>
> not just whole display - but now imagine a table with a screen and touch and 4
> people around it one along each side and multiple windows floating about like
> scraps of paper... just an illustration where you'd want window-by-window
> rotation done by compositor as well.

Sure, but that's complex enough - and difficult enough to reason even
about the desired UI semantics - that it really wants a prototype
first, or even a mockup. How do you define orientation in a table
scenario? If you're doing gesture-based/constant rotation (rather than
quantised to 90°), how do you animate that, and where does the
threshold for relayout lie? Without knowing what to design for, it's
hard to come up with a protocol which makes sense.

Luckily, writing extensions is infinitely less difficult than under
X11, so the general approach has been to farm these out to separate
extensions and then bring them in later if they turn out to make sense
in a global context. The most relevant counter-example (and
anti-pattern) I can think of is XI2 multitouch, where changing things
was so difficult that we had to design from the moon from the get-go.
The result, well, was XI2 multitouch. Not my finest moment, but
luckily I stepped out before it was merged so can just blame Peter.

>> > actually the other way around... clients know where the vkbd region(s) are
>> > so client can shuffle content to be visible. :)
>>
>> In a VKB (rather than overlay-helper, as used for complex composition)
>> scenario, I would expect xdg-shell to send a configure event to resize
>> the window and allow for the VKB. If this isn't sufficient - I
>> honestly don't know what the behaviour is under X11 - then a potential
>> version bump of wl_text could provide for this.
>
> no - resizing is a poorer solution. tried that in x11. first obvious port of
> call. imagine vkbd is partly translucent... you want it to still be over window
> content. imagine a kbd split onto left and right halves, one in the middle of
> the left and right edges of the screen (because screen is bigger). :)

Yeah, gotcha; it does fall apart after more than about a minute's
thought. Seems like this has been picked up though, so happy days.

>> > (pretend a phone with 4 external monitors attached).
>>
>> Hell of a phone. More seriously, yes, a display-management API could
>> expose this, however if the aim is for clients to communicate intent
>> ('this is a presentation') rather than for compositors to communicate
>> situation ('this is one of the external monitors'), then we probably
>> don't need this. wl_output already provides the relative geometry, so
>> all that is required for this is a way to communicate output type.
>
> i was thinking a simplified geometry. then again client toolkits can figure
> that out and present a simplified enum or what not to the app too. but yes -
> some enumerated "type" attached to the output would be very nice. smarter
> clients can decide their intent based on what is listed as available - adapt to
> the situation. dumber ones will just ask  for a fixed type and "deal with it"
> if they don't get it.

I think exposing an output type would be relatively uncontroversial.
The fullscreen request already takes a target output; would that cover
your uses, or do you really need to request initial presentation of
non-fullscreen windows on particular outputs? (Actually, I can see
that: you'd want your PDF viewer's primary view to tack to your
internal output, and its presentation view aimed at the external
output. Jasper/Manuel - any thoughts?)

>> > surfaces should be
>> > able to hint at usage - eg "i want to be on the biggest tv". "i want to be
>> > wherever you have a small mobile touch screen" etc. compositor deals with
>> > deciding where they would go based on the current state of the world
>> > screen-wise and app hints.
>>
>> Right. So if we do have this client-intent-led interface (which would
>> definitely be the most Wayland-y approach), then we don't need to
>> advertise output types and wl_output already deals with the rest, so
>> no change required here?
>
> well the problem here is the client is not aware of the current situation. is
> that output on the right a tv on the other side of the room, ore a projector, or
> perhaps an internal lcd panel? is it far from the user or touchable (touch
> surface). if it's touchable the app may alter ui (make buttons bigger - remove
> scrollbars to go into a touch ui mode as opposed ro mouse driven...). maybe app
> is written for multitouch controls specifically and thus a display far from the
> user with a single mouse only will make the app "useless"? app should be able
> to know what TYPE of display it is on - what types are around and be able to
> ask for a type (may or may not get it). important thing is introducing the
> concept of a type and attaching it to outputs (and hints on surfaces).

Touchable is something to think about for sure, but right now we
really don't have a way of exposing the compositor's internal mapping
of input device co-ordinate space to global/output/surface co-ordinate
space. This is something that needs solving for tablets anyway (does
it span the whole desktop as a faux-relative device, or have you bound
it 1:1 to a particular output/surface as an absolute device?), so is
definitely not going to drop off the radar.

>>   - client informs compositor it is capable of managing the rotation process
>>   - compositor triggers beginning of rotation and informs client
>>   - client renders a series of frames animating between initial and final
>> states
>>   - last frame is produced twice: one rotated in-client (i.e. output
>> displaying in non-rotated co-ordinate space will show the buffer as
>> rotated), one in newly-rotated co-ordinate space
>
> ahh for the fullscreen case :) i'm thinking multiple windows at once - so
> compositor would rotate the surface output at composite time given an angle
> the client gives it. client would resize buffer each time with the "anchor"
> being at window center (need to define that actually somehow). client now just
> loops from 0 to 90 degrees, with a newly sized buffer each time and newly
> rendered content. you're definitely thinking the fullscreen case subset here
> where you are indeed right. :)

If that's not synchronised, it'll look like crap. I think the most
productive thing would be to knock together a closed system (in the
engineering, rather than freeness, sense) which implements this well,
and see what we can pull out of that generically.

>>   - client attaches and commits pre-rotated frame (the first)
>>   - client immediately attaches and commits rotated-coord-buffer (the
>> second), tagged with information that this is the final frame and the
>> compositor should atomically switch its display along with displaying
>> this buffer
>>
>> This is relatively complex, and introduces a whole lot of failure
>> modes: having a 'pending rotation' will be tricky to get exactly right
>> without proper testcases, and we're also relying on clients to not
>> hold our entire display pipeline hostage, so we'd need some kind of
>> arbitrary timeout too I guess. It is entirely doable, but the
>> complexity makes me think that this would best be proven either as an
>> external extension, or something that was prepared to live out-of-tree
>> for a little while whilst it was proven.
>
> oh sure. we'd end up doing this kind of thing regardless - in xdg and wayland
> core or not. if not - we'd do an extn and have it all working etc. - then maybe
> come back for "hey - this stuff works - see? how about we standardize and all
> agree on the same core protocol for this now?"

Yes! This is exactly how the system is supposed to work. :)

Sounds like we're pretty much on the same page, and do have a couple
of concrete points picked out. Progress!

Cheers,
Daniel