[Spice-devel] [RFC PATCH spice 0.8 01/19] server/spice.h: semi-seamless migration interface, RHBZ #738266

[Alon Levy]

On Mon, Sep 19, 2011 at 04:06:25PM +0300, Yonit Halperin wrote:
> On 09/19/2011 03:15 PM, Gerd Hoffmann wrote:
> >On 09/19/11 11:46, Yonit Halperin wrote:
> >>semi-seamless migration details:
> >>
> >>migration source side
> >>---------------------
> >>(1) spice_server_migrate_info (*): tell client to link
> >>to the target side - send SPICE_MSG_MAIN_MIGRATE_BEGIN.
> >>client_migrate_info cmd is asynchronous.
> >>(2) Complete client_migrate_info only when the client has been connected
> >>to the target - wait for
> >>SPICE_MSGC_MAIN_MIGRATE_(CONNECTED|CONNECT_ERROR) or a timeout.
> >>(3) spice_server_migrate_end: tell client migration it can switch to
> >>the target - send
> >>SPICE_MSG_MAIN_MIGRATE_END.
> >>(4) client cleans up all data related to the connection to the source
> >>and switches to the target.
> >>It sends SPICE_MSGC_MAIN_MIGRATE_END.
> >
> >So the switch-host message will not be used any more? How do you handle
> >compatibility to spice servers which don't support this migration scheme?
> >
> Older qemu/spice-server will still use the switch host message and
> the spice client still support it.
> With respect to spice-severs that doesn't support this scheme:
> qemu checks #ifdef SPICE_INTERFACE_MIGRATION, and acts accordingly
> (calls switch-host or migrate_end).
> The spice client responds according to the server:
> if <=Rhel6.0 : seamless migration
> if Rhel6.1: Switch host
> if Rhel6.2: semi-seamless

We need to handle another case:
 New client.
 New source server.
 Old target server.

Server and client both have the SEMI_SEAMLESS cap, so server uses semi-seamless,
client connects to target, but target doesn't support semi seamless and proceeds
to send updates as usual without waiting for a SPICEC_MSG_MIGRATE_END.

Two possible solutions:
 1. client_migrate_info - add an extra parameter for "target version". Spice server will
 see the target is too old, and use switch host. Requires change to libvirt, so
 ignored for now.
 2. When client connects to the server it sees that the server doesn't have the SEMI_SEAMLESS_CAP
 present. So it disconnects, and sends to the source a SPICEC_MSG_MIGRATE_TARGET_CONNECTION_FAILED_TARGET_TOO_OLD
, or maybe just a generic error message (if we have something like that) and use that as the reason.
 The server sees this, and proceeds to do the old switch host behavior. This doesn're require libvirt changes,
 it does create two connections to the target, but at least the vm is never stopped during that time.

> 
> Each scheme has its own code pathway in the client.
> 
> >This looks a bit like seamless migration half-done. What is missing to
> >full seamless migration support? The spice channels need to transfer the
> >state via client. Also qemu on the source needs to wait for that to
> >complete I guess? Anything else?
> We are still missing keeping the target stopped till the target
> spice server is restored from the data passed through the client.
> Thanks to the fix Alon sent for RHBZ #729621, we could block the
> target from starting using a vm state change handler, but it is a
> workaround...
> IMHO, if we could have async notifiers for migration before it
> starts and just before it completes (as we had in Rhel5), that would
> have been the best solution (it will also allow us not to use
> client_migrate_info for the pre-connection to the target). By async
> notifiers I mean
> (1) notifiers that hold the migration process till they complete.
> (2) to add notifications (a) before migration starts (today we
> receive one only after it starts), (b) before migration completes -
> when the src and target are both stopped (today we don't have
> control over the target state).
> 
> >
> >>enum {
> >>SPICE_MIGRATE_CLIENT_NONE = 1,
> >>SPICE_MIGRATE_CLIENT_WAITING,
> >>SPICE_MIGRATE_CLIENT_READY,
> >>};
> >
> >Is that still needed?
> >
> Don't think so.
> 
> >>+/* migration interface */
> >>+#define SPICE_INTERFACE_MIGRATION "migration"
> >>+#define SPICE_INTERFACE_MIGRATION_MAJOR 1
> >>+#define SPICE_INTERFACE_MIGRATION_MINOR 1
> >>+typedef struct SpiceMigrateInterface SpiceMigrateInterface;
> >>+typedef struct SpiceMigrateInstance SpiceMigrateInstance;
> >>+typedef struct SpiceMigrateState SpiceMigrateState;
> >>+
> >>+struct SpiceMigrateInterface {
> >>+ SpiceBaseInterface base;
> >>+ void (*migrate_info_complete)(SpiceMigrateInstance *sin);
> >>+};
> >>
> >>+struct SpiceMigrateInstance {
> >>+ SpiceBaseInstance base;
> >>+ SpiceMigrateState *st;
> >>+};
> >
> >Why a new interface? The only purpose it seems to serve is adding a
> >single callback. It is registered unconditionally by the spice core code
> >in qemu. I fail to see why we can't just add the callback to the core
> >interface. That would save a bunch of code to handle the new interface.
> >
> >I think we should try to make sure the qemu <-> spice-server migration
> >API can handle both semi-seamless and real seamless client migration.
> I added a new interface since I think that for supporting seamless
> migration we may need more callbacks (e.g., if the above async
> notifiers will be implemented). IMHO, it is more readable then
> adding it to the core interface.
> >Then the two spice servers can negotiate themself which method they
> >pick, without qemu being involved here. I think the minimum we'll need
> >here is a second notification (when spice channel state migration via
> >slient is finished), either using a second callback or by adding a
> >reason argument to the one we have. But maybe just blocking in
> >spice_server_migrate_end() works too (the guest is stopped at that point
> >anyway, so going async doesn't buy us that much here ...).
> >
> >What is the plan for the master branch? How will this work with
> >multiclient enabled?
> >
> For one client - the same.
> For multiclient - not sure yet, maybe for small number of clients do
> the same, and for the rest fall back to switch_host? (with the risk
> of losing the ticket).
> >cheers,
> >Gerd
> >
>