[PATCH 0/5] QEMU VFIO live migration

[Gonglei (Arei)]

> -----Original Message-----
> From: Zhao Yan [mailto:yan.y.zhao at intel.com]
> Sent: Thursday, February 21, 2019 12:08 PM
> To: Gonglei (Arei) <arei.gonglei at huawei.com>
> Cc: cjia at nvidia.com; kvm at vger.kernel.org; aik at ozlabs.ru;
> Zhengxiao.zx at Alibaba-inc.com; shuangtai.tst at alibaba-inc.com;
> qemu-devel at nongnu.org; kwankhede at nvidia.com; eauger at redhat.com;
> yi.l.liu at intel.com; eskultet at redhat.com; ziye.yang at intel.com;
> mlevitsk at redhat.com; pasic at linux.ibm.com; felipe at nutanix.com;
> Ken.Xue at amd.com; kevin.tian at intel.com; dgilbert at redhat.com;
> alex.williamson at redhat.com; intel-gvt-dev at lists.freedesktop.org;
> changpeng.liu at intel.com; cohuck at redhat.com; zhi.a.wang at intel.com;
> jonathan.davies at nutanix.com
> Subject: Re: [PATCH 0/5] QEMU VFIO live migration
> 
> On Thu, Feb 21, 2019 at 03:33:24AM +0000, Gonglei (Arei) wrote:
> >
> > > -----Original Message-----
> > > From: Zhao Yan [mailto:yan.y.zhao at intel.com]
> > > Sent: Thursday, February 21, 2019 9:59 AM
> > > To: Gonglei (Arei) <arei.gonglei at huawei.com>
> > > Cc: alex.williamson at redhat.com; qemu-devel at nongnu.org;
> > > intel-gvt-dev at lists.freedesktop.org; Zhengxiao.zx at Alibaba-inc.com;
> > > yi.l.liu at intel.com; eskultet at redhat.com; ziye.yang at intel.com;
> > > cohuck at redhat.com; shuangtai.tst at alibaba-inc.com;
> dgilbert at redhat.com;
> > > zhi.a.wang at intel.com; mlevitsk at redhat.com; pasic at linux.ibm.com;
> > > aik at ozlabs.ru; eauger at redhat.com; felipe at nutanix.com;
> > > jonathan.davies at nutanix.com; changpeng.liu at intel.com;
> Ken.Xue at amd.com;
> > > kwankhede at nvidia.com; kevin.tian at intel.com; cjia at nvidia.com;
> > > kvm at vger.kernel.org
> > > Subject: Re: [PATCH 0/5] QEMU VFIO live migration
> > >
> > > On Thu, Feb 21, 2019 at 01:35:43AM +0000, Gonglei (Arei) wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Zhao Yan [mailto:yan.y.zhao at intel.com]
> > > > > Sent: Thursday, February 21, 2019 8:25 AM
> > > > > To: Gonglei (Arei) <arei.gonglei at huawei.com>
> > > > > Cc: alex.williamson at redhat.com; qemu-devel at nongnu.org;
> > > > > intel-gvt-dev at lists.freedesktop.org; Zhengxiao.zx at Alibaba-inc.com;
> > > > > yi.l.liu at intel.com; eskultet at redhat.com; ziye.yang at intel.com;
> > > > > cohuck at redhat.com; shuangtai.tst at alibaba-inc.com;
> > > dgilbert at redhat.com;
> > > > > zhi.a.wang at intel.com; mlevitsk at redhat.com; pasic at linux.ibm.com;
> > > > > aik at ozlabs.ru; eauger at redhat.com; felipe at nutanix.com;
> > > > > jonathan.davies at nutanix.com; changpeng.liu at intel.com;
> > > Ken.Xue at amd.com;
> > > > > kwankhede at nvidia.com; kevin.tian at intel.com; cjia at nvidia.com;
> > > > > kvm at vger.kernel.org
> > > > > Subject: Re: [PATCH 0/5] QEMU VFIO live migration
> > > > >
> > > > > On Wed, Feb 20, 2019 at 11:56:01AM +0000, Gonglei (Arei) wrote:
> > > > > > Hi yan,
> > > > > >
> > > > > > Thanks for your work.
> > > > > >
> > > > > > I have some suggestions or questions:
> > > > > >
> > > > > > 1) Would you add msix mode support,? if not, pls add a check in
> > > > > vfio_pci_save_config(), likes Nvidia's solution.
> > > > > ok.
> > > > >
> > > > > > 2) We should start vfio devices before vcpu resumes, so we can't rely
> on
> > > vm
> > > > > start change handler completely.
> > > > > vfio devices is by default set to running state.
> > > > > In the target machine, its state transition flow is
> running->stop->running.
> > > >
> > > > That's confusing. We should start vfio devices after vfio_load_state,
> > > otherwise
> > > > how can you keep the devices' information are the same between source
> side
> > > > and destination side?
> > > >
> > > so, your meaning is to set device state to running in the first call to
> > > vfio_load_state?
> > >
> > No, it should start devices after vfio_load_state and before vcpu resuming.
> >
> 
> What about set device state to running in load_cleanup handler ?
> 

The timing is fine, but you should also think about if should set device state 
to running in failure branches when calling load_cleanup handler.

Regards,
-Gonglei