[Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Hans de Goede]

Hi,

On 01/12/2011 06:40 PM, Anthony Liguori wrote:
> On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
>>
>> Hi folks,
>>
>> Looks like the spicevmc patch kicked the guest qagent discussion, so lets start with this, although it isn't related much to the agent issue
>> itself ...
>>
>>
>> The spicevmc chardev just pipes data from a chardev user within qemu
>> to libspice and adds a "type" tag to it so libspice knows now to wind
>> up the other end. There are several types:
>>
>> (1) vdagent, the spice guest agent. Will discuss this in detail
>> below.
>>
>> (2) smartcard, this basically pipes the smartcard protocol over spice.
>> Patches for smartcard support are on the list and should be almost
>> ready for merge now. If you want connect a remote smart card reader
>> to your guest you can use a tcp chardev, which will build a data
>> pipeline like this:
>>
>> ccid-passthrough <-> tcp chardev <-> tcp protocol <->
>> vcsclient <-> libcacard
>>
>> Or you can use the spicevmc chardev to use spice as transport:
>>
>> ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
>> spice client <-> libcacard
>>
>> If someone comes up with a vnc extention one could create something
>> simliar for vnc tunneling:
>>
>> ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
>> gtk-vnc widget <-> libcacard
>>
>> (3) usb forwarding. Hans is busy with this. No working code yet.
>> Will probably work pretty simliar to smartcard.
>>
>> (4) termial forwarding. Just an idea right now. Nowdays that the spice
>> client side moves to gtk it would be easy to embed a termial widget,
>> therby allowing easy access to the serial console using something
>> like this:
>>
>> -chardev spicevmc,id=console,type=terminal
>> -device isa-serial,index=0,chardev=console
>>
>> So even if you put the guest agent discussion completely aside the
>> spicevmc chardev clearly has its uses.
>>
>>
>> Ok, now for the spice vdagent. Alon posted the link to the specs in
>> the spice wiki already, here it is again:
>> http://spice-space.org/page/Whiteboard/AgentProtocol
>>
>> The header file with the protocol specification comes with the
>> spice-protocol package, here is a direct link:
>> http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
>>
>> Linux agent code is here:
>> http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
>>
>> The protocol can be multiplexed via VDIChunkHeader->port. spice
>> actually does that as the mouse messages (VDAgentMouseState) are
>> generated by the spice server itself. Everything else is just piped
>> from the guest to the spice client (and back). The protocol also has
>> capabilities (VDAgentAnnounceCapabilities).
>>
>> There isn't much spice-specific stuff in there. The clipboard bits
>> for example should work unmodified with vnc, one would just have to
>> hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
>> client support of course).
>
> VNC already supports copy/paste as part of the protocol so can the agent protocol be terminated in QEMU such that the server can make use of the standard protocol extensions?
>

That depends on if the VNC copy/paste support was done right. With right
I mean that it should consist of the following messages:
1) A clipboard grab (send by guest -> client if ctrl+c pressed inside
    the guest, other direction if ctrl-c is pressed in any app on the
    client machine.) This should include a list of supported types the
    app now owning the clipboard can offer the clipboard in. For example
    text/plain-utf-8, text/html
2) A clipboard release send when the clipboard owning app exits
3) A clipboard request, send by one side when it wants to get the
    clipboard data, prereq: the other side owns the clipboard, the
    request is for a type in the list of types advertised when grabbing
4) clipboard data, response to clipboard request, could also be a nack
    when a release / request message cross each other.

This is basically how both windows and X11 work wrt copy and paste at
the low level, and by far the sanest way to support guest <-> client
os copy paste (event driven, no polling or some such involved).

I agree with the notion of having 2 agents in the guest, 1 system
wide and 1 per session. Assuming we get a standardized protocol for
the per session agent and we can somehow fit vnc's existing copy and
paste support into the above model, it should be able to use that.

While on the 2 agent topic. As I've just completed written the
spice agent for Linux I've some experience on this. Currently the
spice agent actually already consists of 2 processes an always
running daemon and a per X-session agent, there are 2 reasons for this:
1) Thinks like the paravirt mouse are best done at the system level
2) A multiplexer is needed between the client -> session agent channel,
and the actual session agents, as multiple session agents could be
running (think switch user functionality), and the messages should be
forwarded to the session agent in the active (focussed / showing)
session. spice-vdagentd uses console kit to determine the active
session and monitor session changes.

This does not really impact any protocol designs, but still it is
something to keep in mind.

>>
>> I think the only spice-specific bit in the protocol is the display
>> enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
>> messages. The numbers there simply reference the spice display
>> channel number of the display in question, which just doesn't exist
>> outside spice. Of course one can just ignore that for now as there is
>> no multihead support outside spice anyway ...
>>
>>
>> Also related: paravirtual mouse. I'd suggest to go for something new,
>> based on virtio-serial, doing just the mouse and nothing else.
>
> I'd agree. I think we want something that actually terminates in the kernel for Linux guests since then we can expose it as an evdev device. No special X driver would be needed.

A paravirt mouse would need some sort of guest os support, yes. But
not necessarily in the kernel. currently spice-vdagentd uses uinput
to create an evdev device and send events from userspace. And the spice
agent under windows does the same (although there the events are
generated by the per user session agent process).

I'm not saying this should not be in the kernel, just that it does
not have to be in the kernel. It might even make sense to try and
use such a ipc mechanism for this, that in one guest os it could be
a kernel driver and in the other a userspace solution, but I'm not
sure how feasible that it.

> Is this something that makes sense for Spice in the future?

Yes, as long as guest support is kept in mind. Looking at it from
that pov. I think I might favor a userspace solution, as we likely
need an agent anyways, and adding / doing drivers for some guest
os-s is a pain.

>
>>
>> The VDAgentMouseState messages have one problem: They send the pointer
>> position as-is, which introduces a dependency on the screen size.

Yeah, if we could get rid of that, that would be great. We could even
introduce a new mouse message type to the existing spice vdagent protocol
and use capabilities to switch between the 2.

>> Spice deals with it, and although that is a little bit ugly it isn't a
>> big issue either as spice knows how big the screens are.

Well the client knows, but a guest os system level agent does not
necessarily know. With the current linux spice agent, the system level
daemon depends on the per session agent to notify it of resolution changes
(and if there is no per session agent active, the mouse gets dropped
  into server mode). So I'm in favor of fixing this even within the
current spice specific vdagent protocol

 >> When using
>> this outside spice context this becomes nasty though.
>>
>> Also I think it is a bad idea to mix guest agent stuff with the
>> paravirtual mouse. The mouse events need to get routed to the
>> X-Server, whereas the agent stuff is handled by some other daemon.
>> Also the multiplexing the spice server has to do to inject the mouse
>> messages supports that view IMHO.
>>
>> Related note: It is probably a good idea to also separate stuff which
>> is handled best by a system-wide guest daemon (such as reboot
>> requests, package installs, whatever, vdagent doesn't do that kind of
>> management btw) and stuff which needs to be handled by a per-user
>> daemon which has access to the user's X-Session (clipboard, ...).
>
> Definitely. VMware uses two separate daemons--one that runs in a system context and one that runs per user X session.
>

+1

>>
>> Ok, how to go forward now? Here is what I think we should do:
>>
>> (1) Merge spicevmc chardev. Needed anyway for compatibility and
>> for the other use cases outlined above.
>> (2) Design a paravirtual mouse device. Make sure it fullfills the
>> spice requirements, which are basically:
>> * support multiple displays.
>> * don't eat tons of cpu like usb-tablet does.
>> (3) spice can use the new pvmouse now and depricate sending mouse
>> events via vdagent protocol.
>> (4) In parallel review the vdagent protocol, then either extend or
>> redesign it for other use cases.
>
> I'm okay with this as long as there is a commitment from the Spice team to convert on a common framework.

I (*) am in favor of moving to a common framework. As long as
compatibility is maintained with the old agent, but that all lives inside
spice server, so should not be an issue.

*) Speaking for myself not the team (not that I know of anyone in the
team who is against).

> I'd like to have a little more discussion about agent design first to make sure that we're on the same page.
>
> For instance, Spice makes use of a 1-off protocol whereas something like virt-agent uses an established RPC protocol (XML-RPC). I'm not tied to using any particular protocol, but I think it's very important to use a standardized, well specified protocol.

I'm not sure what something like XML-RPC buys us here, other then
dragging in a lot of extra dependencies. I'm not saying that I'm against
using xmlrpc but I'm not sold on it either.

Note while on the subject of design, I think that having some sort of
capabilities negotiation so that we can provide compatibility between
different versions is important.

Regards,

Hans