[Telepathy] Vino-telepathy for a single window?

[Robert McQueen]

Dafydd Harries wrote:
> Presumably it's using XGetImage, which doesn't handle occluded regions
> properly. It's possible to do this properly using XComposite + XDamage.
> XComposite lets you redirect the drawing offscreen so you can copy the
> contents regardless of occlusion, and XDamage will tell you when it changes so
> you can update.

This is a Vino thing rather than a Telepathy thing, but it would be
bloody useful... :D

We actually looked into this at the beginning of the year at LCA - we'd
also quite like the ability to share one application because it
alleviates the need to modify each and every application to support
collaborative stuff, and addresses (some of) the privacy/security
concerns of whole-desktop sharing.

I believe libvncserver (inside Vino) already uses damage and composite
internally, to get a pixbuf for the root window. So I think its not too
hard to modify it to look at one particular window, but the devil is in
the details. There are a lot of them. :(

* One application is not just one window - any menu or dialog box is
  another window which needs to be tracked and shared. Dialogs are
  pretty easy as they're transients for their parent, and alternative
  top-level windows will have the same WM_CLASS/etc or binary/PID hints
  if they belong to the same instance.

  Menus are a total pain in the ass because currently, they're not
  hinted or parented in such a way that makes it obvious which
  application they belong to. You could probably patch Gtk and Qt to do
  this in future, but it wouldn't help for current apps, so you'd need
  to do some heuristic to track menu windows which float over apps...

* How you actually represent these multiple windows to the other end is
  unclear. Do you composite a new desktop together, which has only the
  windows believed to belong to the given application? What do you then
  do with window decorations? Do you have to provide a separate window
  manager-like functionality for each shared application, so the remote
  users can rearrange the windows (imagine GIMP). Or do you have to make
  the single-app desktop laid out the same as the sharing user's
  desktop? Or instead do you create one RFB framebuffer for each window
  belonging to the app, and then extend the protocol to send the
  hints/type/etc for each window so it gets decorated properly at the
  far end?

* Input delivery is also... fun. Because currently, X input doesn't
  support co-ordinate redirection, so input events are absolute. You get
  the problem that if an application's window is covered by another at
  that absolute screen position, you can't deliver remotely-originated
  events to the right application.

* If an application is minimised, usually the window manager unmaps it,
  which means it stops being rendered by the application. If an
  application is locally minimised, but remotely still visible, the
  window manager needs to keep it mapped even if it's not being
  composited onto the screen.

* There are also security issues - if you share an app then minimise it,
  switch to another desktop, or wander off, the other person then has
  fairly arbitrary and unsupervised access to your system via that
  application. At a first approximation we probably need to set the
  DEMANDS_ATTENTION hint on applications which are being used by 3rd
  parties, but we probably also need to hook into screen
  locking/screensaver stuff too.

So because of these issues, and partly for efficiency's sake, then I
think this kind of single-app-sharing functionality would need to be
done with the collusion of a compositing window manager, to track which
windows belong to which application, map them off-screen so they
actually render, make sure the user is aware what is being shared, and
arrange them off-screen in an appropriate non-overlapping way so you can
transform and deliver input events.

We thought about having a row of hackergotchies/avatars in the title bar
to show who was using a certain application would be pretty cool. Bonus
points if you then use MPX and have one pointer belonging to each user
who is using the application. Lose points for the application being
utterly confused at that at the moment however. :)

Tackling this is somewhere on our wishlist, but we don't have enough
tuits at the moment (round, square or otherwise).

Regards,
Rob

-- 
Robert McQueen                                 +44 7876 562 564
Director, Collabora Ltd.             http://www.collabora.co.uk