[Spice-devel] [Qemu-devel] seamless migration with spice

[Yonit Halperin]

Hi,
On 03/15/2012 04:23 PM, Hans de Goede wrote:
> Hi,
>
> On 03/15/2012 03:07 PM, Yonit Halperin wrote:
>> Hi,
>> On 03/15/2012 02:36 PM, Hans de Goede wrote:
>>> Hi,
>>>
>>> On 03/15/2012 01:11 PM, Yonit Halperin wrote:
>>>> On 03/13/2012 09:40 AM, Gerd Hoffmann wrote:
>>>>> Hi,
>>>>>
>>>>>> It is not easy when you have 2 components, and it is much less easy
>>>>>> when
>>>>>> you have 3 or 4 components. So why make it more complicated if you
>>>>>> can
>>>>>> avoid it. Especially since there is no functional reason for
>>>>>> making the
>>>>>> qemu/client capabilities/versions dependent on the server internal
>>>>>> data.
>>>>>
>>>>> qemu has ways to handle compatibility in the vmstate format. We can
>>>>> use
>>>>> those capabilities. That of course requires exposing the structs to be
>>>>> saved to qemu and adds some complexity to the qemu<-> spice interface.
>>>>>
>>>>> What session state is needed by the target?
>>>>> What of this can be negotiated between client and target host without
>>>>> bothering the source?
>>>>> What needs be transfered from source to target, either directly or via
>>>>> client?
>>>>>
>>>>>>> If this is a hard requirement then using the vmstate channel isn't
>>>>>>> going
>>>>>>> to work. The vmstate is a one-way channel, no way to negotiate
>>>>>>> anything
>>>>>>> between source and target.
>>>>>>>
>>>>>> We can do this via the client.
>>>>>
>>>>> Then you can send the actual state via client too.
>>>>> Out-of-band negotiation for the blob send via vmstate scares me.
>>>>>
>>>>> Can we please start with a look at which state we actually have to
>>>>> send
>>>>> over?
>>>> Ok, I can take the display and sound channels.
>>>> Alon, can you take the smartcard?
>>>> Hans, spicevmc?
>>>
>>> Easy, the spicevmc channel has no state which needs to be migrated,
>>> except
>>> for things in the red_channel_client base class:
>>>
>>> 1) Partially received spice messages
>>> 2) Possible pending pipe items when writes from qemu -> client have
>>> blocked.
>>>
>>> I assume that the red_channel_client base class will handle migrating
>>> them,
>>> if we migrate them at all.
>>>
>>> Instead of migrating we could:
>>> For 1. expect the client to stop sending new messages at a certain point
>>> during the migration, and ensure we've processed any pending messages
>>> after this point.
>>>
>>> For 2. we could flush pending items and set a flag to stop channel
>>> implementations from queuing new ones, at which point for spicevmc the
>>> data will get queued inside qemu and migrating it no longer is
>>> a spice-server problem to migrate it (and we need migration support for
>>> the data possibly queued inside qemu anyways).
>>>
>> We have an implementation for this: After migration had completed,
>> each spice-server channel sent MSG_MIGRATE to the corresponding client
>> channel. The msg was sent after all the pending msgs to the client had
>> already been sent.
>> In response, the client sent SPICE_MSGC_MIGRATE_FLUSH_MARK to the
>> server, after it completed sending all its pending messages.
>
> Yes, that is exactly what I thought we did but I was too lazy to check the
> source :) So that would mean that other then assuring no data gets
> queued up in spicevmc after sending the MSG_MIGRATE to the client (see
> below),
> no changes are needed to spicevmc, as it is essentially stateless.
>
Indeed, however, in order for this to work, we still need to make sure 
the source qemu is not killed before we complete this.

Cheers,
Yonit.

>> Then the "blob" data transfer and completion of socket switching has
>> occurred.
>>
>> Regarding the usb data in the server that should be flushed to qemu:
>> we need to save it after the source vm is stopped. So I think it is
>> too late for flushing it to qemu, unless you refereed to the special
>> vmstate we will have for spice, if we go in that solution direction.
>
> I was talking about the other direction, so data queued in qemu which
> should be flushed
> to the server (and then forwarded to the client). IOW what I mean is that
> after the spice-server channel sent MSG_MIGRATE, it should no longer
> read data
> from the qemu-chardev, even if it receives a chardev wakeup from qemu,
> leaving the
> data inside qemu to be migrated using qemu's standard migration mechanism.
>
> Regards,
>
> Hans