[Spice-devel] [PATCH 00/12] seamless mode support

Tue Aug 8 23:23:17 UTC 2017

Hi

----- Original Message -----
> Thanks for the feedback!
> 
> On Tue, Aug 8, 2017 at 3:44 PM Marc-André Lureau <
> marcandre.lureau at redhat.com> wrote:
> 
> > Hi
> >
> > ----- Original Message -----
> > > Hey all,
> > >
> > > this series introduces very basic seamless (aka unity aka coherence, ...)
> > > mode
> > > support for spice-gtk and both Windows and Linux vdagent.
> > > Many other rival companies have already added this feature to their
> > products,
> > > therefore it might be the right time to cover it too...
> >
> > Do you know if virtualbox is actively maintaining that feature? I mean
> > does it work well with gnome 3 & windows 10 for example? It could be worth
> > to ask if there are ideas or code we could share for this feature perhaps.
> >
> > I've tried VirtualBox with Windows 10 and it doesn't work very well, the
> new apps from Windows Store are not visible at all.
> Here's part of their implementation for Win, most important is
> the VBoxEnumFunc() that decides which window should be shown.
> https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Additions/WINNT/VBoxTray/VBoxSeamless.cpp
> Maybe it could be useful when targeting older versions of windows,
> otherwise I'm not sure, haven't look at the linux version yet.
> 
> > Agent sends list of visible areas/change in visible area whenever a window
> > > is created, moved or closed/hidden. The client then masks out the areas
> > > that shoud not be visible accordingly.
> >
> > How do you deal with non-rectangular apps/windows?
> 
> 
> Currently all windows are considered to be rectangular.
> Is there any well-known program that has non-rectangular windows? Perhaps
> some clock widgets?

Typically, windows 7, gnome/gtk+ 3 apps have rounded window borders. Since it will probably keep the window decoration of the guest, it will have rendering glitches on the corners. It's even more complicated if you consider transparency of course (again the per-window drawing context is necessary, so composition can be done properly by the client).

> >
> >
> >
> > > The spice-gtk and  linux agent implementation is heavily based on older
> > work
> > > from Ondřej Holý and Lukáš Venhoda, so big shout-out for them.
> > > I merged the initial commits in spice-gtk and fixed some minor issues.
> > > You can have a look at the original version here, if you wish to,
> > > on branch "seamless-mode":
> > >
> > > spice-protocol: git://people.freedesktop.org/~lvenhoda/spice-protocol
> > > spice-gtk:      git://people.freedesktop.org/~lvenhoda/spice-gtk
> > > linux vdagent:  git://people.freedesktop.org/~lvenhoda/vd_agent
> > >
> > > Further progress can be tracked on my Github,
> > > again branch "seamless-mode" (might not be always stable):
> > >
> > > spice-protocol: https://github.com/jjanku/spice-protocol.git
> > > spice-gtk:      https://github.com/jjanku/spice-gtk.git
> > > linux vdagent:  https://github.com/jjanku/linux-vd_agent.git
> > > win32 vdagent:  https://github.com/jjanku/win32-vd_agent.git
> > >
> > > DEMO:
> > > If everything goes well, you should hopefully see output similar to this:
> > > https://www.youtube.com/watch?v=IX49z8VbD-c
> > > (windows leaving traces have been fixed since then)
> >
> > Nice, easy to see a few glitches though. I suppose it's even worse with a
> > more recent windows/os. Also I suppose you have bigger glitch when windows
> > overlap each others. Ideally, each windows would have a different drawing
> > context/buffer, so composition/decomposition on the client side wouldn't
> > suffer glitches. This is something we could more easily have with
> > virgl-enabled guest, since each application has different rendering
> > contexts already.
> >
> >
> What do you mean by glitches? The traces after moving/resizing the
> minesweeper window?

Yes. There will be more glitches if it creates independent windows, when going to overview or switching between windows etc. I suppose you don't at the moment, right? (iirc, virtualbox doesn't create a client window per guest window either)

> > TESTING:
> > > Has been tested on:
> > > Fedora 25 with Gnome, Win7, Win8.1, Win10; all with basic preinstalled
> > > programs
> > > (there seem to be some extra windows that pass the filter on Win8.1,
> > >  other versions should work just fine)
> > >
> > > NOTICE:
> > > spice-gtk: commit ba87ba2: removes drawing function that was in the
> > original
> > > implementation and I found it redundant, it works fine without it for me.
> > > If you have any problems with transparency, delete this commit and
> > > give it another try, otherwise it shall be merged with the initial
> > commit.
> > >
> > > LIMITATIONS & FUTURE PLANS:
> > > - window integration could be better (currently one window that stays on
> > > top),
> > >   maybe create some dummy windows that would control the actual windows
> > on
> > >   remote system? not sure really, any ideas?
> >
> > Indeed, the app focus should be managed by the client.
> 
> 
> > > - multi monitor/workspace support
> > > - launching programs: currently no way for linux, Start menu on Windows
> >
> > I'd consider this a secondary goal
> >
> > > - drag & drop (started some research for Win, see below if you're
> > interested)
> >
> > same here.
> >
> > Better drag and drop would also benefit "non-rootless" remote desktop. I
> > have done some work in the past around this, you can see some old wip here:
> >
> >
> > https://github.com/elmarco/spice-gtk/commit/c3eba974975d4f4d8fefc5b5f2db59db33c49fec
> >
> > https://github.com/elmarco/vdagent-gtk/commit/4b58c752e6844d3a8d763964f36c9e9d29929d36
> >
> > (based on an incomplete vdagent rewrite:
> > https://github.com/elmarco/vdagent-gtk)
> >
> > Note I was only using Gtk, so code should remain portable.
> >
> 
> Thanks, I'll have a look. Since it isn't just seamless-mode dependent, I'd
> like to address it rather sooner than later.

Yes, it would be a nice thing to have.

>From my memories, clipboard&dnd is a lot of difficult code to write if we don't use gtk+ (x11/wayland/windows all have different APIs, quite different in design).

Even today, afaik, copy/paste doesn't work with wayland, right? Using gtk+ would bring portability and simplify the code. But there are some complications to make a real portable vdagent, since windows & linux agents work a bit differently. Can we overcome them? I have no idea what's more important between rootless apps & dnd. For sure, having a portable vdagent would save a lot of effort.

The most enviable dnd imho is for files (although there is the xfer messages for now). This is even trickier as the destination usually needs an url to copy/move etc the files from. The idea I had was to use a hidden webdav folder, with links to client files created during dnd operation. (it was one of the reason why I wrote the webdav channel)

> > > - clicking through: gdk_window_input_shape_combine_region() is not
> > supported
> > >   on Win, gdk_window_shape_combine_region() might do the trick
> > >   (as it does on some linux systems), not tested though
> > > - Wayland support?
> >
> > That should be a priority at this point. Using and fixing gtk+ where
> > necessary should give the portability needed.
> >
> > > - windows in Gnome have no border/shadow, they blend into the background,
> > >   maybe we should draw shadows/borders for them in spice-gtk? (low
> > priority)
> >
> > To be discussed with gtk/desktop team, I suppose there is a way to
> > activate shadow.
> >
> > You probably don't want the window decoration & border being drawn by the
> > client though.
> >
> > > - Windows: switch to in-context event hooks? (low priority)
> >
> > What's that?
> >
> 
> https://msdn.microsoft.com/en-us/library/windows/desktop/dd318524(v=vs.85).aspx
> It should be faster, but would probably require some sort of inter-process
> communication to get the data back to vdagent. VirtualBox implementation
> uses these. Delayed events don't seem to be the current issue though.
> 
> > - any suggestions are welcomed :) The points above should be discussed.
> > >
> > > Related: https://www.spice-space.org/page/Features/RootlessWindows
> >
> > To be honest, I would focus on getting the feature to work with a modern
> > linux guest first (modifying it as necessary), rather than targeting an old
> > windows.
> >
> > The demo is older and a bit misleading, here's a new one:
> https://youtu.be/JENQCkKq4sA
> On Win10 there seems to be the problem that messages with window positions
> are ahead of display data, but the identification of windows themselves

vdagent and display are on different socket/streams, without synchronization. An idea would be to use the display channel for window messages. It may help with those timing glitches, if somehow the events are synchronized with the drawing by the guest.

> works. I agree that it should support modern linux guest, however the
> ability to run windows apps seamlessly in linux environment is quite
> desired, isn't it?

Eh, probably as desirable for a Windows client using a Linux guest :)