[Spice-devel] [PATCH 00/12] seamless mode support

[Marc-André Lureau]

Hi

----- Original Message -----
> Hey all,
> 
> this series introduces very basic seamless (aka unity aka coherence, ...)
> mode
> support for spice-gtk and both Windows and Linux vdagent.
> Many other rival companies have already added this feature to their products,
> therefore it might be the right time to cover it too...

Do you know if virtualbox is actively maintaining that feature? I mean does it work well with gnome 3 & windows 10 for example? It could be worth to ask if there are ideas or code we could share for this feature perhaps.

> Agent sends list of visible areas/change in visible area whenever a window
> is created, moved or closed/hidden. The client then masks out the areas
> that shoud not be visible accordingly.

How do you deal with non-rectangular apps/windows?

> 
> The spice-gtk and  linux agent implementation is heavily based on older work
> from Ondřej Holý and Lukáš Venhoda, so big shout-out for them.
> I merged the initial commits in spice-gtk and fixed some minor issues.
> You can have a look at the original version here, if you wish to,
> on branch "seamless-mode":
> 
> spice-protocol: git://people.freedesktop.org/~lvenhoda/spice-protocol
> spice-gtk:      git://people.freedesktop.org/~lvenhoda/spice-gtk
> linux vdagent:  git://people.freedesktop.org/~lvenhoda/vd_agent
> 
> Further progress can be tracked on my Github,
> again branch "seamless-mode" (might not be always stable):
> 
> spice-protocol: https://github.com/jjanku/spice-protocol.git
> spice-gtk:      https://github.com/jjanku/spice-gtk.git
> linux vdagent:  https://github.com/jjanku/linux-vd_agent.git
> win32 vdagent:  https://github.com/jjanku/win32-vd_agent.git
> 
> DEMO:
> If everything goes well, you should hopefully see output similar to this:
> https://www.youtube.com/watch?v=IX49z8VbD-c
> (windows leaving traces have been fixed since then)

Nice, easy to see a few glitches though. I suppose it's even worse with a more recent windows/os. Also I suppose you have bigger glitch when windows overlap each others. Ideally, each windows would have a different drawing context/buffer, so composition/decomposition on the client side wouldn't suffer glitches. This is something we could more easily have with virgl-enabled guest, since each application has different rendering contexts already.

> TESTING:
> Has been tested on:
> Fedora 25 with Gnome, Win7, Win8.1, Win10; all with basic preinstalled
> programs
> (there seem to be some extra windows that pass the filter on Win8.1,
>  other versions should work just fine)
> 
> NOTICE:
> spice-gtk: commit ba87ba2: removes drawing function that was in the original
> implementation and I found it redundant, it works fine without it for me.
> If you have any problems with transparency, delete this commit and
> give it another try, otherwise it shall be merged with the initial commit.
> 
> LIMITATIONS & FUTURE PLANS:
> - window integration could be better (currently one window that stays on
> top),
>   maybe create some dummy windows that would control the actual windows on
>   remote system? not sure really, any ideas?

Indeed, the app focus should be managed by the client.

> - multi monitor/workspace support
> - launching programs: currently no way for linux, Start menu on Windows

I'd consider this a secondary goal

> - drag & drop (started some research for Win, see below if you're interested)

same here.

Better drag and drop would also benefit "non-rootless" remote desktop. I have done some work in the past around this, you can see some old wip here:

https://github.com/elmarco/spice-gtk/commit/c3eba974975d4f4d8fefc5b5f2db59db33c49fec
https://github.com/elmarco/vdagent-gtk/commit/4b58c752e6844d3a8d763964f36c9e9d29929d36

(based on an incomplete vdagent rewrite: https://github.com/elmarco/vdagent-gtk)

Note I was only using Gtk, so code should remain portable.
 
> - clicking through: gdk_window_input_shape_combine_region() is not supported
>   on Win, gdk_window_shape_combine_region() might do the trick
>   (as it does on some linux systems), not tested though
> - Wayland support?

That should be a priority at this point. Using and fixing gtk+ where necessary should give the portability needed.

> - windows in Gnome have no border/shadow, they blend into the background,
>   maybe we should draw shadows/borders for them in spice-gtk? (low priority)

To be discussed with gtk/desktop team, I suppose there is a way to activate shadow.

You probably don't want the window decoration & border being drawn by the client though.

> - Windows: switch to in-context event hooks? (low priority)

What's that?

> - any suggestions are welcomed :) The points above should be discussed.
> 
> Related: https://www.spice-space.org/page/Features/RootlessWindows

To be honest, I would focus on getting the feature to work with a modern linux guest first (modifying it as necessary), rather than targeting an old windows.

> 
> ----
> DRAG & DROP for Win:
> Quick search has revealed three potential solutions so far
> (I haven't tested any of them):
> 
> [1] Send WM_DROPFILES
> - seems simple and straightforward, but outdated and not very well supported,
>   probably not the way to go
> 
> [2] Register IDropSource and simulate mouse movement
> + no hassle with finding the correct window user wanted to drop files into
> - might result in poor user experience because of the mouse simulation
> - DoDragDrop() blocks calling thread until the drop is finished
> 
> [3] Find the window for drag&drop ourselves,
>     inject a DLL into the process that owns the given window,
>     get IDropTarget and call Drop()
> + completely in background
> - need to find the correct window

I think there is no need for any of that if you use gtk. I haven't tried to work on this recently, but hopefully you can get some ideas from my wip branches above.

Good work so far though!

cheers