Wayland talk at FOSDEM

[Philipp Kerling]

Hey Pekka,

thanks for your input - further comments below. The talk has been
accepted, but shortened to 25 min due to the amount of talks that had
been submitted overall. This means that I will have significantly less
time than I thought and most of the advanced stuff will have to go I'm
afraid, but I'll still try to give a comprehensive overview within the
timeframe I'm given.

2017-12-08 (金) の 10:39 +0200 に Pekka Paalanen さんは書きました:
> On Thu, 07 Dec 2017 23:56:49 +0200
> Philipp Kerling <pkerling at casix.org> wrote:
> 
> > Hi Pekka,
> > 
> > 2017-12-07 (木) の 09:56 +0200 に Pekka Paalanen さんは書きました:
> > > On Tue, 05 Dec 2017 16:46:41 +0200
> > > Philipp Kerling <pkerling at casix.org> wrote:
> > >   
> > > > Hey everyone,
> > > > 
> > > > I've been thinking about submitting a Wayland-themed talk to
> > > > the
> > > > FOSDEM
> > > > graphics devroom [1] and wanted to check with the community if
> > > > anyone
> > > > else has considered this or another Wayland topic and whether
> > > > you
> > > > think
> > > > it would be a good idea™.
> > > > 
> > > > What I would want to talk about is a beginner's introduction to
> > > > Wayland
> > > > from the client perspective. I think that resources on this
> > > > (and on
> > > > Wayland in general to be honest) are quite scarce and that I
> > > > could
> > > > pass
> > > > on some of the knowledge I have gained by implementing native
> > > > Wayland
> > > > support in Kodi this way.  
> > > 
> > > Hi,
> > > 
> > > that sounds like a good idea.  
> > 
> > Thanks!
> > 
> > >   
> > > > This would include stuff such as:
> > > >  * Wayland architecture
> > > >     * Comparison with X
> > > >     * Security architecture
> > > >     * Limitations: What is not possible with Wayland
> > > > (currently)  
> > > 
> > > Please underline that many things are wanted and the only reason
> > > they
> > > are not there yet is because they have not been worked on enough
> > > to
> > > produce a Waylandish solution and reach a concensus, not because
> > > they
> > > are somehow "banned". :-)  
> > 
> > Good point since you hear from time to time that Wayland "forbids"
> > this-and-that and therefore is evil ... I'll definitely keep it in
> > mind. Even so, I guess it is a quite safe bet to e.g. say that
> > accessing the buffer contents of all other applications without any
> > user authentication will not be a feature of mainstream Wayland
> > compositors anytime soon.
> 
> Hi Philipp
> 
> Yes. There are some more or less banned things, but I would tend to
> think that they are a minority in numbers. "Without user permission"
> is
> the main problem for most of the popular features that are not yet
> standardised - the features are good, necessary and supportable, but
> the main issue is how to expose them without compromising security
> and
> without annoying users or asking users questions they cannot reliably
> answer.
> 
> I believe it is actually quite likely that the solution to the
> authentication problem is mostly outside of the scope of Wayland
> protocol, but we would still need a working example in some
> environment
> to be able to comfortably promote "privileged interfaces".
> 
> > > In some cases, the Waylandish solution to produce an end-user
> > > feature
> > > may be very different from what has traditionally been done. In
> > > some
> > > other cases, a solution might be better or already exist outside
> > > of
> > > Wayland.  
> > 
> > Yes, the problem lies mostly in the expectation that things will
> > work
> > just like they did with X.
> > 
> > >   
> > > >     * Difference between core and extension protocols  
> > > 
> > > This is a bit hazy to me as well, depending on at what level you
> > > are
> > > looking to use Wayland. Only wl_display, wl_registry and
> > > wl_callback
> > > interfaces are not extensions, strictly speaking. What extensions
> > > are
> > > mandatory depends on the target environment. Technically, e.g.
> > > wl_compositor is an extension and not mandatory, but it's hard to
> > > find
> > > a use case without it.  
> > 
> > I'd have argued that the wayland.xml shipped with wayland is the
> > "core"
> > protocol and wayland-protocols + all other stuff are "extensions."
> > Is
> > that a reasonable assumption or should I be more careful about the
> > wording here? In any case, I guess it makes sense to mention that
> > Wayland is not only built for traditional desktop environments,
> > even if
> > that is what I'll be focusing on in the talk.
> 
> The split is not so clear. The most obvious example of the contrary
> is
> wl_shell extension which is in wayland.xml, but is not considered
> "core". E.g. IVI environment using Wayland does not use it.
I think I've raised this point in #wayland before, but part of the
problem seems to be that wayland.xml does not explicitly spell out (in
most cases) what is considered "core" at the moment and what not. You
can sort of infer it from the description, but it's not very clear imo.
As you said though, protocols depend heavily on the environment Wayland
is used in. What do you in general think about making a recommendation
such as "All compositors MUST implement wl_compositor, wl_shm, etc.
Desktop compositors SHOULD additionally implement this and that."? That
would at least give client apps some information on what they can
expect to be there.

> wl_shell is a "shell extension", which is a category of Wayland
> extensions. With Wayland core protocol and wl_compositor, you cannot
> actually show a window. A shell extension is necessary to tell the
> compositor what it should do with a wl_surface, i.e. assign the
> wl_surface a role. Wayland.xml does have some interfaces which can be
> considered core and assign surface roles, but they are not enough to
> have a working application. An example of such role is the pointer
> cursor.
> 
> The other global interfaces (advertised through wl_registry) in
> wayland.xml are quite "core" I think, but it's not unthinkable to
> imagine to replace them in the future, should the need arise.
> 
> That is actually a design feature of Wayland: (almost) everything is
> an
> extension. As a corollary, everything can be deprecated and replaced
> with something else if really necessary, forming a new "core". The
> old
> stuff can eventually be dropped completely, except for the interfaces
> that exist without having to find them in wl_registry.
> 
> For instance, wl_output could be replaced with something better, but
> the replacement would be a hard one because wl_surface references
> wl_output and practically everything references wl_surface. It would
> not be easy to drop wl_output. So I guess that could be one
> definition
> of "core", what you could replace without essentially throwing out
> everything we have.
Yeah, that could be a working definition. With that notion of "core"
protocols, what would your wording be for the distinction between
wayland.xml as shipped with wayland vs. stuff in wayland-protocols/all
external protocols not part of wayland.xml?

> However, for your presentation, I imagine you would like to settle
> for a
> simplification. Too many details will cloud the message. So it would
> be
> fine to say that core is wayland.xml without wl_shell, for instance.
> Maybe also add that not all of core is mandatory for all environments
> either.
Sounds OK. It should be fine if I get the message across that it is
really environment-dependent and desktop WM usage is just one of the
possible environments (the one I'll talk about though). That makes it
clear that the scope of Wayland is broader and might explain some of
the peculiarities of the protocol.

> Technically the definition of an extension is "it is advertised
> through
> wl_registry".
> 
> > > So it kind of depends on your definition of "Wayland". Purely
> > > IPC? A
> > > graphical output protocol? A windowing protocol with input? A
> > > desktop
> > > window system protocol?
> > > 
> > > Maybe giving a definition for "Wayland" would be a good way to
> > > start?  
> > 
> > Is there an officially accepted one? :-)
> > My personal stance would be: Windowing protocol definition +
> > ecosystem
> > (C libraries, scanner, extension protocols that among other things
> > cover the traditional desktop use case)
> 
> That could be a nice definition.
> 
> I suppose what's on the website is all what we officially have. The
> meaning of the name depends quite much on the context it is used in.
> 
> I think that a big part of Wayland is the architecture: desktop
> environment and domain specific display servers, and applications
> doing
> the drawing themselves; libwayland being essentially just an IPC
> shim.
> Moving window management and compositing into the display server.
> 
> For the Wayland architecture to work to begin with, there have been
> huge amounts of prerequisite work, kernel drivers with standardized
> UAPIs for a start: DRM, KMS, evdev; not forgetting EGL and Mesa and
> libinput, and kernel features like dmabuf. It has been a long road
> away
> from display server specific hardware drivers.
You're quite right about that. It's easy to forget that a few years ago
it was generally accepted that everything is done by specialized X11
drivers. I might include this thought as a side-mark if time allows.

> 
> The thing almost everyone new to Wayland gets wrong is they think
> that
> Wayland is a display server. This leads to strange questions like
> "how
> do I run wayland?", "where does wayland save logs?", etc. and of
> course
> blaming "wayland" for bugs or misfeatures that could actually be
> anywhere
> except Wayland. (Protocol issues OTOH are Wayland issues.)
The more I read the more I get the feeling that explaining the
architecture alone will take the better half of the talk. Hope I can
make it as concise as possible and still have it understandable.

> The idea of a separate display server in each DE project is still
> fairly alien to the masses. Everyone has had just Xorg for so long
> they
> don't even realize that it's not the only X11 server either.
It's not? :o)

> 
> 
> > > >  * Relevant compositors to test on and how to use nested mode
> > > >  * Basic client programming
> > > >     * Protocols needed to get a surface on screen
> > > > (wl_compositor,
> > > >       wl_surface, wl_shm, wl_shm_pool, wl_shell,
> > > > wl_shell_surface)  
> > > 
> > > Mind that wl_shell is finally officially deprecated with the
> > > stabilization of xdg-shell.  
> > 
> > I'm not sure how to handle that yet. Technically, it would make
> > sense
> > to directly introduce xdg-shell and skip the wl_shell details. But
> > realistically speaking it will still take some time until there is
> > sufficient compositor support and you can get by with an
> > application
> > that does not support anything but the xdg_wm_base global.
> > One alternative could be to introduce xdg-shell unstable v6 instead
> > of
> > wl_shell and tell people to switch to xdg-shell stable at some
> > point?
> > I'm not sure the latest KDE stable release supports xdg-shell
> > unstable
> > v6 though. Last time I checked KDE only had v5 in stable.
> 
> Yeah, the timing is perfectly awkward here. ;-)
> 
> If you want to give hands-on examples of how to write the code, then
> I'd recommend xdg-shell unstable v6. The migration to the stable
> version is almost a sed job, I believe.
Yup, recently done it for Kodi. Real C code would be too bulky to show
during the talk, but maybe I'll show protocol interactions. I'll do
that with unstable v6 then and briefly mention that one can switch to
the stable version easily once it's out there in the compositors.

> 
> v5 is more problematic because of naming conflicts with the stable
> version.
> 
> > > I suppose one detail might be worth mentioning: client projects
> > > are
> > > expected to run wayland-scanner (or equivalent) as part of their
> > > builds. The only exception are the interfaces in wayland.xml, but
> > > it may be a surprise to realize that wayland.xml is the
> > > exception,
> > > not
> > > the rule.  
> > 
> > Thinking about it, there is probably more that should be mentioned,
> > like common pitfalls. From the back of my mind:
> >  * Don't forget to flush... (I learned this the hard way)
> 
> Yes, flushing should be part of the client main loop, it's even in
> the
> prepare_read API doc. :-)
> 
> If someone is sprinkling flush calls in other places, it is highly
> likely that they have something fundamentally broken. Maybe their
> main / event loop is broken, or they are not returning to it as often
> as they should. This applies to clients, but especially to servers,
> IMO.
Depends. Kodi e.g. uses a dedicated thread to process Wayland events,
which means you have to flush if you plan to wait for something. We can
debate the wiseness of that design, but not in this mail :-) It does
work quite well atm.

> >  * Don't create lots of objects that have no destructor request
> > (like
> >    wl_registry)
> >  * Be careful with multithreading
> >  * Use the prepare_read API for poll/read
> 
> That seems to be the most important. There are quite some details
> about
> threading, but maybe they would be too much for the time you can have
> for the presentation. :-)
Definitely...

> Maybe also mention that libwayland-server is completely thread-
> unsafe,
> contrary to libwayland-client which has some thread-safety support.
> 
> Well, everything that can be traced back to the same server
> wl_display
> C struct/object must be processed/poked in the same thread. There is
> no
> static data IIRC, but since the C objects heavily reference each
> other
> and there is zero locking, it's pretty safe to just say it's
> thread-unsafe.
At the moment and especially given the new time constraints, I think I
won't mention libwayland-server at all.

> 
> Thanks,
> pq

Thank you!
Philipp