[Spice-devel] [Qemu-devel] seamless migration with spice

[Hans de Goede]

Hi,

On 03/15/2012 03:07 PM, Yonit Halperin wrote:
> Hi,
> On 03/15/2012 02:36 PM, Hans de Goede wrote:
>> Hi,
>>
>> On 03/15/2012 01:11 PM, Yonit Halperin wrote:
>>> On 03/13/2012 09:40 AM, Gerd Hoffmann wrote:
>>>> Hi,
>>>>
>>>>> It is not easy when you have 2 components, and it is much less easy
>>>>> when
>>>>> you have 3 or 4 components. So why make it more complicated if you can
>>>>> avoid it. Especially since there is no functional reason for making the
>>>>> qemu/client capabilities/versions dependent on the server internal
>>>>> data.
>>>>
>>>> qemu has ways to handle compatibility in the vmstate format. We can use
>>>> those capabilities. That of course requires exposing the structs to be
>>>> saved to qemu and adds some complexity to the qemu<-> spice interface.
>>>>
>>>> What session state is needed by the target?
>>>> What of this can be negotiated between client and target host without
>>>> bothering the source?
>>>> What needs be transfered from source to target, either directly or via
>>>> client?
>>>>
>>>>>> If this is a hard requirement then using the vmstate channel isn't
>>>>>> going
>>>>>> to work. The vmstate is a one-way channel, no way to negotiate
>>>>>> anything
>>>>>> between source and target.
>>>>>>
>>>>> We can do this via the client.
>>>>
>>>> Then you can send the actual state via client too.
>>>> Out-of-band negotiation for the blob send via vmstate scares me.
>>>>
>>>> Can we please start with a look at which state we actually have to send
>>>> over?
>>> Ok, I can take the display and sound channels.
>>> Alon, can you take the smartcard?
>>> Hans, spicevmc?
>>
>> Easy, the spicevmc channel has no state which needs to be migrated, except
>> for things in the red_channel_client base class:
>>
>> 1) Partially received spice messages
>> 2) Possible pending pipe items when writes from qemu -> client have
>> blocked.
>>
>> I assume that the red_channel_client base class will handle migrating them,
>> if we migrate them at all.
>>
>> Instead of migrating we could:
>> For 1. expect the client to stop sending new messages at a certain point
>> during the migration, and ensure we've processed any pending messages
>> after this point.
>>
>> For 2. we could flush pending items and set a flag to stop channel
>> implementations from queuing new ones, at which point for spicevmc the
>> data will get queued inside qemu and migrating it no longer is
>> a spice-server problem to migrate it (and we need migration support for
>> the data possibly queued inside qemu anyways).
>>
> We have an implementation for this: After migration had completed,
> each spice-server channel sent MSG_MIGRATE to the corresponding client channel. The msg was sent after all the pending msgs to the client had already been sent.
> In response, the client sent SPICE_MSGC_MIGRATE_FLUSH_MARK to the server, after it completed sending all its pending messages.

Yes, that is exactly what I thought we did but I was too lazy to check the
source :)  So that would mean that other then assuring no data gets
queued up in spicevmc after sending the MSG_MIGRATE to the client (see below),
no changes are needed to spicevmc, as it is essentially stateless.

> Then the "blob" data transfer and completion of socket switching has occurred.
>
> Regarding the usb data in the server that should be flushed to qemu: we need to save it after the source vm is stopped. So I think it is too late for flushing it to qemu, unless you refereed to the special vmstate we will have for spice, if we go in that solution direction.

I was talking about the other direction, so data queued in qemu which should be flushed
to the server (and then forwarded to the client). IOW what I mean is that
after the spice-server channel sent MSG_MIGRATE, it should no longer read data
from the qemu-chardev, even if it receives a chardev wakeup from qemu, leaving the
data inside qemu to be migrated using qemu's standard migration mechanism.

Regards,

Hans