[Spice-devel] spice-gtk: Synchronized/duplicated input across multiple widgets

[Victor Toso]

Hi,

On Tue, Mar 30, 2021 at 11:41:59AM -0400, James Harvey wrote:
> I think a very useful addition to spice would be
> synchronized/duplicated input, so one physical input can be given
> simultaneously to multiple spice servers.
> 
> The most common use for this might be system administration.  An admin
> would be able to connect to multiple machines, and perform common
> tasks on all of them at once, such as upgrading software, manual
> malware scans, changing settings, etc.

That means that all VMs should be very similar, in the same
state... seems a bit error prone to me.

> This exists in a few other programs.  There are ssh clients that do
> this, but personally I use tmux's synchronized panes feature for this
> after I've started different ssh connections in them.  I'm not aware
> of any linux GUI remote clients with this feature, and the only
> Windows GUI remote clients I'm aware of with it are BlueStacks and
> Nox, which are android emulators built upon VM software.

ssh is a shell in the remote machine. It can have different
failures but the scope is smaller than spice, I think.

> Obviously, it is on the user to ensure input works as intended.
> If servers have different resolutions, layouts, icon
> placements, if one hasn't finished a previous task, etc,
> identical input may not have identical behavior in all clients.
> 
> Before understanding the extent of what spice-gtk handles, I
> approached the remmina developers, largely because one instance
> of their client can already handle multiple connections in tabs
> or separate windows.  Now understanding that spice-gtk receives
> the physical mouse input and sends it to the spice server, it
> looks to me by far that the easiest way to do this is make a
> chance in spice-gtk and then have the clients do whatever is
> necessary to inform the widgets to enter/exit this mode.
> 
> In tmux, the "setw synchronize-panes" command toggles
> synchronizing input among all panes in a window, so giving
> input to any of them is duplicated in the others.  There's no
> master pane that has to be typed into.
> 
> In the Windows android emulators BlueStacks and Nox, their
> synchronized input feature allows you on a remote connection
> window to select other running connections to receive
> synchronized input from that window.  The one you start from is
> the master window, which you have to give input to for it to be
> duplicated.  If you give input into one of the other windows
> you've selected to receive synchronization,
> the input is only given to that window, which is useful if a window
> becomes unsynchronized to easily bring it back to where the others are
> without needing to toggle the overall synchronization or turn
> including that window off.
> 
> I think it would be nice if a spice client could tell spice-gtk
> to enter/exit either of these modes: synchronized input among a
> group of spice-gtk widgets; or synchronized input made in a
> master spice-gtk widget to be given to a number of spice-gtk
> widgets.
> 
> I think a restriction that spice-gtk widgets can only
> synchronize with other widgets running on the same machine
> would simplify implementation and security concerns.  Perhaps
> the spice developers would want only widgets running within the
> same process to be able to synchronize.  In that case, remmina
> might be most easily able to implement this feature, but other
> clients could add the ability to run multiple connections
> within a single process to be able to implement it as well.
> 
> Is this something someone would like to implement?  If not, is
> it something that someone would be willing to mentor, giving me
> locations of existing code that likely need changes, etc?  And
> if I could get it working, that the feature would likely be
> accepted as a pull request rather than denied as an unwanted
> feature?

IMHO a better approach would be a guest component that handles
the operations that you are looking for. It might seem a bit
bigger project but spice already has a port channel that can make
client <-> guest communication such as we do with shared folder
feature.

Actually, I recall that this kind of project was explored before
by Fleet commander developers [0], not sure if that's what they
have done.

Anyway, spice-gtk tries to keep the same approach as any other
desktop application so that means spice-gtk only receive input
when the widget has the keyboard grabbed (like when you click on
it). Working around this would be bad in general, from security
POV or simply too hard (e.g: in Wayland).

I think the best way forward with your idea would be some sort of
multicast with usb redirection, so you can connect a usb keyboard
to multiple vms. You probably can work on some POC without spice,
by having two QEMU configured with usb-redir over tcp and connect
to them with usbredirect [1]. After you get that working (most
patches here would be in usbredir I think) it would be much
easier or even trival (no changes) in spice-gtk to do what you
want.

[0] https://fleet-commander.org/documentation.html
[1] https://gitlab.freedesktop.org/spice/usbredir/-/merge_requests/2

Cheers,
Victor
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/spice-devel/attachments/20210330/f63e5550/attachment.sig>