[PATCH v4] drm/i915/gvt: Change KVMGT as self load module
He, Min
min.he at intel.com
Fri Dec 7 07:47:59 UTC 2018
Zhenyu,
Overall I think the impact to AcrnGT is not big, we can modify our code to adapt
to the new interfaces.
But I have some comments embedded.
> -----Original Message-----
> From: Zhenyu Wang [mailto:zhenyuw at linux.intel.com]
> Sent: Thursday, December 6, 2018 12:28 PM
> To: Yuan, Hang <hang.yuan at intel.com>; Wang, Zhi A
> <zhi.a.wang at intel.com>; Zhang, Xiong Y <xiong.y.zhang at intel.com>; He,
> Min <min.he at intel.com>
> Cc: intel-gfx at lists.freedesktop.org; Alex Williamson
> <alex.williamson at redhat.com>; intel-gvt-dev at lists.freedesktop.org
> Subject: Re: [PATCH v4] drm/i915/gvt: Change KVMGT as self load module
>
>
> Ping for any more comments? Or ack?
>
> Adding Min, for head-up notify, I've moved 'kvmgt' into self load
> module instead of loaded by i915/gvt, in order to clean up the
> dependence, so need to load 'kvmgt.ko' to enable GVT through
> VFIO/mdev. Hypervisor module needs to call new register/unregister
> function to initialize hypervisor specific interface for GVT. If
> AcrnGT rebase on this, it may need to change initial setup too, so
> could you double check?
>
> thanks
>
> On 2018.12.04 10:40:28 +0800, Zhenyu Wang wrote:
> > This trys to make 'kvmgt' module as self loadable instead of loading
> > by i915/gvt device model. So hypervisor specific module could be
> > stand-alone, e.g only after loading hypervisor specific module, GVT
> > feature could be enabled via specific hypervisor interface, e.g VFIO/mdev.
> >
> > So this trys to use hypervisor module register/unregister interface
> > for that. Hypervisor module needs to take care of module reference
> > itself when working for hypervisor interface, e.g for VFIO/mdev,
> > hypervisor module would reference counting mdev when open and release.
> >
> > This makes 'kvmgt' module really split from GVT device model. User
> > needs to load 'kvmgt' to enable VFIO/mdev interface.
> >
> > v4:
> > - fix checkpatch warning
> >
> > v3:
> > - Fix module reference handling for device open and release. Unused
> > mdev devices would be cleaned up in device unregister when module
> unload.
> >
> > v2:
> > - Fix kvmgt order after i915 for built-in case
> >
> > Cc: Alex Williamson <alex.williamson at redhat.com>
> > Signed-off-by: Zhenyu Wang <zhenyuw at linux.intel.com>
> > ---
> > drivers/gpu/drm/i915/Makefile | 1 +
> > drivers/gpu/drm/i915/gvt/Makefile | 1 -
> > drivers/gpu/drm/i915/gvt/gvt.c | 107 +++++++++++----------------
> > drivers/gpu/drm/i915/gvt/gvt.h | 6 +-
> > drivers/gpu/drm/i915/gvt/hypercall.h | 7 +-
> > drivers/gpu/drm/i915/gvt/kvmgt.c | 21 +++++-
> > drivers/gpu/drm/i915/gvt/mpt.h | 3 +
> > drivers/gpu/drm/i915/intel_gvt.c | 9 ---
> > 8 files changed, 72 insertions(+), 83 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/Makefile
> b/drivers/gpu/drm/i915/Makefile
> > index 0ff878c994e2..ae0d975a6f34 100644
> > --- a/drivers/gpu/drm/i915/Makefile
> > +++ b/drivers/gpu/drm/i915/Makefile
> > @@ -195,3 +195,4 @@ endif
> > i915-y += intel_lpe_audio.o
> >
> > obj-$(CONFIG_DRM_I915) += i915.o
> > +obj-$(CONFIG_DRM_I915_GVT_KVMGT) += gvt/kvmgt.o
> > diff --git a/drivers/gpu/drm/i915/gvt/Makefile
> b/drivers/gpu/drm/i915/gvt/Makefile
> > index b016dc753db9..271fb46d4dd0 100644
> > --- a/drivers/gpu/drm/i915/gvt/Makefile
> > +++ b/drivers/gpu/drm/i915/gvt/Makefile
> > @@ -7,4 +7,3 @@ GVT_SOURCE := gvt.o aperture_gm.o handlers.o vgpu.o
> trace_points.o firmware.o \
> >
> > ccflags-y += -I$(src) -I$(src)/$(GVT_DIR)
> > i915-y += $(addprefix $(GVT_DIR)/,
> $(GVT_SOURCE))
> > -obj-$(CONFIG_DRM_I915_GVT_KVMGT) += $(GVT_DIR)/kvmgt.o
> > diff --git a/drivers/gpu/drm/i915/gvt/gvt.c
> b/drivers/gpu/drm/i915/gvt/gvt.c
> > index a5b760b7bc10..e1c9c20918af 100644
> > --- a/drivers/gpu/drm/i915/gvt/gvt.c
> > +++ b/drivers/gpu/drm/i915/gvt/gvt.c
> > @@ -187,52 +187,6 @@ static const struct intel_gvt_ops intel_gvt_ops = {
> > .write_protect_handler = intel_vgpu_page_track_handler,
> > };
> >
> > -/**
> > - * intel_gvt_init_host - Load MPT modules and detect if we're running in
> host
> > - *
> > - * This function is called at the driver loading stage. If failed to find a
> > - * loadable MPT module or detect currently we're running in a VM, then
> GVT-g
> > - * will be disabled
> > - *
> > - * Returns:
> > - * Zero on success, negative error code if failed.
> > - *
> > - */
> > -int intel_gvt_init_host(void)
> > -{
> > - if (intel_gvt_host.initialized)
> > - return 0;
> > -
> > - /* Xen DOM U */
> > - if (xen_domain() && !xen_initial_domain())
> > - return -ENODEV;
> > -
> > - /* Try to load MPT modules for hypervisors */
> > - if (xen_initial_domain()) {
> > - /* In Xen dom0 */
> > - intel_gvt_host.mpt = try_then_request_module(
> > - symbol_get(xengt_mpt), "xengt");
> > - intel_gvt_host.hypervisor_type =
> INTEL_GVT_HYPERVISOR_XEN;
> > - } else {
> > -#if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT)
> > - /* not in Xen. Try KVMGT */
> > - intel_gvt_host.mpt = try_then_request_module(
> > - symbol_get(kvmgt_mpt), "kvmgt");
> > - intel_gvt_host.hypervisor_type =
> INTEL_GVT_HYPERVISOR_KVM;
> > -#endif
> > - }
> > -
> > - /* Fail to load MPT modules - bail out */
> > - if (!intel_gvt_host.mpt)
> > - return -EINVAL;
> > -
> > - gvt_dbg_core("Running with hypervisor %s in host mode\n",
> > -
> supported_hypervisors[intel_gvt_host.hypervisor_type]);
> > -
> > - intel_gvt_host.initialized = true;
> > - return 0;
> > -}
> > -
> > static void init_device_info(struct intel_gvt *gvt)
> > {
> > struct intel_gvt_device_info *info = &gvt->device_info;
> > @@ -316,7 +270,6 @@ void intel_gvt_clean_device(struct
> drm_i915_private *dev_priv)
> > return;
> >
> > intel_gvt_destroy_idle_vgpu(gvt->idle_vgpu);
> > - intel_gvt_hypervisor_host_exit(&dev_priv->drm.pdev->dev);
> > intel_gvt_cleanup_vgpu_type_groups(gvt);
> > intel_gvt_clean_vgpu_types(gvt);
> >
> > @@ -352,13 +305,6 @@ int intel_gvt_init_device(struct drm_i915_private
> *dev_priv)
> > struct intel_vgpu *vgpu;
> > int ret;
> >
> > - /*
> > - * Cannot initialize GVT device without intel_gvt_host gets
> > - * initialized first.
> > - */
> > - if (WARN_ON(!intel_gvt_host.initialized))
> > - return -EINVAL;
> > -
> > if (WARN_ON(dev_priv->gvt))
> > return -EEXIST;
> >
> > @@ -420,13 +366,6 @@ int intel_gvt_init_device(struct drm_i915_private
> *dev_priv)
> > goto out_clean_types;
> > }
> >
> > - ret = intel_gvt_hypervisor_host_init(&dev_priv->drm.pdev->dev, gvt,
> > - &intel_gvt_ops);
> > - if (ret) {
> > - gvt_err("failed to register gvt-g host device: %d\n", ret);
> > - goto out_clean_types;
> > - }
> > -
> > vgpu = intel_gvt_create_idle_vgpu(gvt);
> > if (IS_ERR(vgpu)) {
> > ret = PTR_ERR(vgpu);
> > @@ -441,6 +380,8 @@ int intel_gvt_init_device(struct drm_i915_private
> *dev_priv)
> >
> > gvt_dbg_core("gvt device initialization is done\n");
> > dev_priv->gvt = gvt;
> > + intel_gvt_host.dev = &dev_priv->drm.pdev->dev;
> > + intel_gvt_host.initialized = true;
> > return 0;
> >
> > out_clean_types:
> > @@ -467,6 +408,44 @@ int intel_gvt_init_device(struct drm_i915_private
> *dev_priv)
> > return ret;
> > }
> >
> > -#if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT)
> > -MODULE_SOFTDEP("pre: kvmgt");
> > -#endif
> > +int
> > +intel_gvt_register_hypervisor(struct intel_gvt_mpt *m)
> > +{
> > + int ret;
> > + void *gvt;
> > +
> > + if (!intel_gvt_host.initialized)
> > + return -ENODEV;
> > +
> > + if (m->type != INTEL_GVT_HYPERVISOR_KVM &&
> > + m->type != INTEL_GVT_HYPERVISOR_XEN)
> > + return -EINVAL;
> > +
> > + /* Get a reference for device model module */
> > + if (!try_module_get(THIS_MODULE))
> > + return -ENODEV;
> > +
> > + intel_gvt_host.mpt = m;
> > + intel_gvt_host.hypervisor_type = m->type;
> > + gvt = (void *)kdev_to_i915(intel_gvt_host.dev)->gvt;
> > +
> > + ret = intel_gvt_hypervisor_host_init(intel_gvt_host.dev, gvt,
> > + &intel_gvt_ops);
I think we can remove the host_init and host_exit interfaces, since now
it's mpt modules who proactively call the GVT-g to initialize and un-initialize,
Thus we can return the intel_gvt_ops in intel_gvt_register_hypervisor() and
moduels initialize its rest part.
Current kvmgt_init-> intel_gvt_register_hypervisor->host_init seems a little bit
redundant.
But it's up to you to make the call to remove it in this patch or other furture
optimization patches.
> > + if (ret < 0) {
> > + gvt_err("Failed to init %s hypervisor module\n",
> > +
> supported_hypervisors[intel_gvt_host.hypervisor_type]);
> > + return -ENODEV;
> > + }
> > + gvt_dbg_core("Running with hypervisor %s in host mode\n",
> > + supported_hypervisors[intel_gvt_host.hypervisor_type]);
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(intel_gvt_register_hypervisor);
> > +
> > +void
> > +intel_gvt_unregister_hypervisor(void)
> > +{
> > + intel_gvt_hypervisor_host_exit(intel_gvt_host.dev);
> > + module_put(THIS_MODULE);
> > +}
> > +EXPORT_SYMBOL_GPL(intel_gvt_unregister_hypervisor);
> > diff --git a/drivers/gpu/drm/i915/gvt/gvt.h
> b/drivers/gpu/drm/i915/gvt/gvt.h
> > index b4ab1dad0143..8a4cf995d755 100644
> > --- a/drivers/gpu/drm/i915/gvt/gvt.h
> > +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> > @@ -52,12 +52,8 @@
> >
> > #define GVT_MAX_VGPU 8
> >
> > -enum {
> > - INTEL_GVT_HYPERVISOR_XEN = 0,
> > - INTEL_GVT_HYPERVISOR_KVM,
> > -};
> > -
> > struct intel_gvt_host {
> > + struct device *dev;
> > bool initialized;
> > int hypervisor_type;
> > struct intel_gvt_mpt *mpt;
> > diff --git a/drivers/gpu/drm/i915/gvt/hypercall.h
> b/drivers/gpu/drm/i915/gvt/hypercall.h
> > index e49a9247ed78..50798868ab15 100644
> > --- a/drivers/gpu/drm/i915/gvt/hypercall.h
> > +++ b/drivers/gpu/drm/i915/gvt/hypercall.h
> > @@ -33,11 +33,17 @@
> > #ifndef _GVT_HYPERCALL_H_
> > #define _GVT_HYPERCALL_H_
> >
> > +enum hypervisor_type {
> > + INTEL_GVT_HYPERVISOR_XEN = 0,
> > + INTEL_GVT_HYPERVISOR_KVM,
> > +};
> > +
> > /*
> > * Specific GVT-g MPT modules function collections. Currently GVT-g
> supports
> > * both Xen and KVM by providing dedicated hypervisor-related MPT
> modules.
> > */
> > struct intel_gvt_mpt {
> > + enum hypervisor_type type;
> > int (*host_init)(struct device *dev, void *gvt, const void *ops);
> > void (*host_exit)(struct device *dev);
> > int (*attach_vgpu)(void *vgpu, unsigned long *handle);
> > @@ -67,6 +73,5 @@ struct intel_gvt_mpt {
> > };
> >
> > extern struct intel_gvt_mpt xengt_mpt;
We can remove this definition, too.
> > -extern struct intel_gvt_mpt kvmgt_mpt;
> >
> > #endif /* _GVT_HYPERCALL_H_ */
> > diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c
> b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > index 1bbd04d30c42..ef248d577e49 100644
> > --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> > +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > @@ -50,6 +50,7 @@
> > #include "gvt.h"
> >
> > static const struct intel_gvt_ops *intel_gvt_ops;
> > +static struct intel_gvt *intel_gvt;
This variable intel_gvt seems useless.
> >
> > /* helper macros copied from vfio-pci */
> > #define VFIO_PCI_OFFSET_SHIFT 40
> > @@ -627,6 +628,12 @@ static int intel_vgpu_open(struct mdev_device
> *mdev)
> > goto undo_iommu;
> > }
> >
> > + /* Take a module reference as mdev core doesn't take
> > + * a reference for vendor driver.
> > + */
> > + if (!try_module_get(THIS_MODULE))
> > + goto undo_group;
> > +
> > ret = kvmgt_guest_init(mdev);
> > if (ret)
> > goto undo_group;
> > @@ -679,6 +686,9 @@ static void __intel_vgpu_release(struct intel_vgpu
> *vgpu)
> > &vgpu->vdev.group_notifier);
> > WARN(ret, "vfio_unregister_notifier for group failed: %d\n", ret);
> >
> > + /* dereference module reference taken at open */
> > + module_put(THIS_MODULE);
> > +
> > info = (struct kvmgt_guest_info *)vgpu->handle;
> > kvmgt_guest_exit(info);
> >
> > @@ -1459,8 +1469,10 @@ static int kvmgt_host_init(struct device *dev,
> void *gvt, const void *ops)
> > struct attribute_group **kvm_vgpu_type_groups;
> >
> > intel_gvt_ops = ops;
> > + intel_gvt = (struct intel_gvt *)gvt;
> > +
> > if (!intel_gvt_ops->get_gvt_attrs(&kvm_type_attrs,
> > - &kvm_vgpu_type_groups))
> > + &kvm_vgpu_type_groups))
Unrelated to this patch.
> > return -EFAULT;
> > intel_vgpu_ops.supported_type_groups = kvm_vgpu_type_groups;
> >
> > @@ -1849,7 +1861,8 @@ static bool kvmgt_is_valid_gfn(unsigned long
> handle, unsigned long gfn)
> > return ret;
> > }
> >
> > -struct intel_gvt_mpt kvmgt_mpt = {
> > +static struct intel_gvt_mpt kvmgt_mpt = {
> > + .type = INTEL_GVT_HYPERVISOR_KVM,
> > .host_init = kvmgt_host_init,
> > .host_exit = kvmgt_host_exit,
> > .attach_vgpu = kvmgt_attach_vgpu,
> > @@ -1868,15 +1881,17 @@ struct intel_gvt_mpt kvmgt_mpt = {
> > .put_vfio_device = kvmgt_put_vfio_device,
> > .is_valid_gfn = kvmgt_is_valid_gfn,
> > };
> > -EXPORT_SYMBOL_GPL(kvmgt_mpt);
> >
> > static int __init kvmgt_init(void)
> > {
> > + if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0)
> > + return -ENODEV;
> > return 0;
> > }
> >
> > static void __exit kvmgt_exit(void)
> > {
> > + intel_gvt_unregister_hypervisor();
> > }
> >
> > module_init(kvmgt_init);
> > diff --git a/drivers/gpu/drm/i915/gvt/mpt.h
> b/drivers/gpu/drm/i915/gvt/mpt.h
> > index c95ef77da62c..9b4225d44243 100644
> > --- a/drivers/gpu/drm/i915/gvt/mpt.h
> > +++ b/drivers/gpu/drm/i915/gvt/mpt.h
> > @@ -360,4 +360,7 @@ static inline bool
> intel_gvt_hypervisor_is_valid_gfn(
> > return intel_gvt_host.mpt->is_valid_gfn(vgpu->handle, gfn);
> > }
> >
> > +int intel_gvt_register_hypervisor(struct intel_gvt_mpt *);
> > +void intel_gvt_unregister_hypervisor(void);
> > +
> > #endif /* _GVT_MPT_H_ */
> > diff --git a/drivers/gpu/drm/i915/intel_gvt.c
> b/drivers/gpu/drm/i915/intel_gvt.c
> > index c22b3e18a0f5..d74e59e22c9d 100644
> > --- a/drivers/gpu/drm/i915/intel_gvt.c
> > +++ b/drivers/gpu/drm/i915/intel_gvt.c
> > @@ -105,15 +105,6 @@ int intel_gvt_init(struct drm_i915_private
> *dev_priv)
> > return -EIO;
> > }
> >
> > - /*
> > - * We're not in host or fail to find a MPT module, disable GVT-g
> > - */
> > - ret = intel_gvt_init_host();
> > - if (ret) {
> > - DRM_DEBUG_DRIVER("Not in host or MPT modules not
> found\n");
> > - goto bail;
> > - }
> > -
> > ret = intel_gvt_init_device(dev_priv);
> > if (ret) {
> > DRM_DEBUG_DRIVER("Fail to init GVT device\n");
> > --
> > 2.19.1
> >
> > _______________________________________________
> > intel-gvt-dev mailing list
> > intel-gvt-dev at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
>
> --
> Open Source Technology Center, Intel ltd.
>
> $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
More information about the intel-gvt-dev
mailing list