[PATCH 6/8] drm/i915/gvt: Introduce vGPU submission ops
Wang, Zhi A
zhi.a.wang at intel.com
Fri Sep 15 06:59:02 UTC 2017
Yes. That's exactly the purpose of the re-factor. :)
Currently before reaching to workload scheduler, the submission layer should do 2 things, one is emulating the HW status, another one is creating vGPU workload, then put the workload on the workload Q.
So both vGuC and vExeclist needs to create vGPU workload and do all kinds of check and shadow on guest context. This part has been moved into scheduler.c with common APIs exported as they will be both used by vGuC and vExeclist
When the scheduler schedules this workload, it calls the submission layer to emulate status of workload and when the workload is completed it calls the submission layer to emulate status of workload, through the callbacks.
So different HW submission emulation layer needs different call backs to emulate HW status. Of course, vGuC wouldn't use vExeclist callbacks here. It needs itss own callbacks to emulate vGuC status for workload schedule-in and schedule-out.
The reason why we have callbacks is scheduler is the actual guy to decide the time
when the FSM in HW submission layer should be moved on.
Thanks,
Zhi.
-----Original Message-----
From: Zhenyu Wang [mailto:zhenyuw at linux.intel.com]
Sent: Friday, September 15, 2017 9:24 AM
To: Wang, Zhi A <zhi.a.wang at intel.com>
Cc: intel-gvt-dev at lists.freedesktop.org
Subject: Re: [PATCH 6/8] drm/i915/gvt: Introduce vGPU submission ops
On 2017.09.15 06:24:01 +0000, Wang, Zhi A wrote:
> Any reason? From my point of view, if we leave workload management stuffs in execlist emulation layer, it should be painful for vGuC since it also needs to do the same job besides emulating GuC...
>
Not sure about vGuC yet, suppose it won't reuse current execlist_emulate_schedule_in/out, but have own emulate logic, so can bring that as common prepare/complete hooks before/after submission, instead of buried inside of workload hooks.
> -----Original Message-----
> From: Zhenyu Wang [mailto:zhenyuw at linux.intel.com]
> Sent: Friday, September 15, 2017 9:13 AM
> To: Wang, Zhi A <zhi.a.wang at intel.com>
> Cc: intel-gvt-dev at lists.freedesktop.org
> Subject: Re: [PATCH 6/8] drm/i915/gvt: Introduce vGPU submission ops
>
> On 2017.09.14 02:50:46 +0800, Zhi Wang wrote:
> > Introduce vGPU submission ops to support easy switching submission
> > mode of one vGPU between different OSes.
> >
> > Signed-off-by: Zhi Wang <zhi.a.wang at intel.com>
> > ---
> > drivers/gpu/drm/i915/gvt/execlist.c | 10 +++++--
> > drivers/gpu/drm/i915/gvt/gvt.h | 14 ++++++++++
> > drivers/gpu/drm/i915/gvt/handlers.c | 16 +++++++++--
> > drivers/gpu/drm/i915/gvt/scheduler.c | 53
> > ++++++++++++++++++++++++++++++++++++
> > drivers/gpu/drm/i915/gvt/scheduler.h | 6 ++++
> > drivers/gpu/drm/i915/gvt/vgpu.c | 13 ++-------
> > 6 files changed, 98 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gvt/execlist.c
> > b/drivers/gpu/drm/i915/gvt/execlist.c
> > index 22c7edf..e608b39 100644
> > --- a/drivers/gpu/drm/i915/gvt/execlist.c
> > +++ b/drivers/gpu/drm/i915/gvt/execlist.c
> > @@ -553,7 +553,7 @@ static void clean_workloads(struct intel_vgpu *vgpu, unsigned long engine_mask)
> > }
> > }
> >
> > -void intel_vgpu_clean_execlist(struct intel_vgpu *vgpu)
> > +void clean_execlist(struct intel_vgpu *vgpu)
> > {
> > enum intel_engine_id i;
> > struct intel_engine_cs *engine;
> > @@ -569,7 +569,7 @@ void intel_vgpu_clean_execlist(struct intel_vgpu *vgpu)
> > }
> > }
> >
> > -int intel_vgpu_init_execlist(struct intel_vgpu *vgpu)
> > +int init_execlist(struct intel_vgpu *vgpu)
> > {
> > enum intel_engine_id i;
> > struct intel_engine_cs *engine;
> > @@ -591,3 +591,9 @@ void intel_vgpu_reset_execlist(struct intel_vgpu *vgpu,
> > for_each_engine_masked(engine, dev_priv, engine_mask, tmp)
> > init_vgpu_execlist(vgpu, engine->id); }
> > +
> > +const struct intel_vgpu_submission_ops intel_vgpu_execlist_submission_ops = {
> > + .name = "execlist",
> > + .init = init_execlist,
> > + .clean = clean_execlist,
> > +};
> > diff --git a/drivers/gpu/drm/i915/gvt/gvt.h
> > b/drivers/gpu/drm/i915/gvt/gvt.h index 7a770b1..c619728 100644
> > --- a/drivers/gpu/drm/i915/gvt/gvt.h
> > +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> > @@ -142,6 +142,17 @@ struct vgpu_sched_ctl {
> > int weight;
> > };
> >
> > +enum {
> > + INTEL_VGPU_EXECLIST_SUBMISSION = 1,
> > + INTEL_VGPU_GUC_SUBMISSION,
> > +};
> > +
> > +struct intel_vgpu_submission_ops {
> > + const char *name;
> > + int (*init)(struct intel_vgpu *vgpu);
> > + void (*clean)(struct intel_vgpu *vgpu); };
>
> I think moving submission layer operation on workload into this ops is also better, e.g prepare/complete, etc.
>
> > +
> > struct intel_vgpu_submission {
> > struct intel_vgpu_execlist execlist[I915_NUM_ENGINES];
> > struct list_head workload_q_head[I915_NUM_ENGINES];
> > @@ -152,6 +163,9 @@ struct intel_vgpu_submission {
> > DECLARE_BITMAP(tlb_handle_pending, I915_NUM_ENGINES);
> > void *ring_scan_buffer[I915_NUM_ENGINES];
> > int ring_scan_buffer_size[I915_NUM_ENGINES];
> > + const struct intel_vgpu_submission_ops *ops;
> > + int virtual_submission_interface;
> > + bool active;
> > };
> >
> > struct intel_vgpu {
> > diff --git a/drivers/gpu/drm/i915/gvt/handlers.c
> > b/drivers/gpu/drm/i915/gvt/handlers.c
> > index 2ee494b..aad6f46 100644
> > --- a/drivers/gpu/drm/i915/gvt/handlers.c
> > +++ b/drivers/gpu/drm/i915/gvt/handlers.c
> > @@ -1480,9 +1480,11 @@ static int elsp_mmio_write(struct intel_vgpu
> > *vgpu, unsigned int offset, static int ring_mode_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
> > void *p_data, unsigned int bytes) {
> > + struct intel_vgpu_submission *s = &vgpu->submission;
> > u32 data = *(u32 *)p_data;
> > int ring_id = render_mmio_to_ring_id(vgpu->gvt, offset);
> > bool enable_execlist;
> > + int ret;
> >
> > write_vreg(vgpu, offset, p_data, bytes);
> >
> > @@ -1504,8 +1506,18 @@ static int ring_mode_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
> > (enable_execlist ? "enabling" : "disabling"),
> > ring_id);
> >
> > - if (enable_execlist)
> > - intel_vgpu_start_schedule(vgpu);
> > + if (!enable_execlist)
> > + return 0;
> > +
> > + if (s->active)
> > + return 0;
> > +
> > + ret = intel_vgpu_select_submission_ops(vgpu,
> > + INTEL_VGPU_EXECLIST_SUBMISSION);
> > + if (ret)
> > + return ret;
> > +
> > + intel_vgpu_start_schedule(vgpu);
> > }
> > return 0;
> > }
> > diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c
> > b/drivers/gpu/drm/i915/gvt/scheduler.c
> > index 544a887..d979c4f 100644
> > --- a/drivers/gpu/drm/i915/gvt/scheduler.c
> > +++ b/drivers/gpu/drm/i915/gvt/scheduler.c
> > @@ -881,6 +881,7 @@ void intel_vgpu_clean_submission(struct
> > intel_vgpu
> > *vgpu) {
> > struct intel_vgpu_submission *s = &vgpu->submission;
> >
> > + intel_vgpu_select_submission_ops(vgpu, 0);
> > i915_gem_context_put(s->shadow_ctx);
> > kmem_cache_destroy(s->workloads);
> > }
> > @@ -935,6 +936,58 @@ int intel_vgpu_setup_submission(struct
> > intel_vgpu
> > *vgpu) }
> >
> > /**
> > + * intel_vgpu_select_submission_ops - select virtual submission
> > +interface
> > + * @vgpu: a vGPU
> > + * @interface: expected vGPU virtual submission interface
> > + *
> > + * This function is called when guest configures submission interface.
> > + *
> > + * Returns:
> > + * Zero on success, negative error code if failed.
> > + *
> > + */
> > +int intel_vgpu_select_submission_ops(struct intel_vgpu *vgpu,
> > + unsigned int interface)
> > +{
> > + struct intel_vgpu_submission *s = &vgpu->submission;
> > + const struct intel_vgpu_submission_ops *ops[] = {
> > + [INTEL_VGPU_EXECLIST_SUBMISSION] =
> > + &intel_vgpu_execlist_submission_ops,
> > + };
> > + int ret;
> > +
> > + if (WARN_ON(interface >= ARRAY_SIZE(ops)))
> > + return -EINVAL;
> > +
> > + if (s->active) {
> > + s->ops->clean(vgpu);
> > + s->active = false;
> > + gvt_dbg_core("vgpu%d: de-select ops [ %s ] \n",
> > + vgpu->id, s->ops->name);
> > + }
> > +
> > + if (interface == 0) {
> > + s->ops = NULL;
> > + s->virtual_submission_interface = 0;
> > + gvt_dbg_core("vgpu%d: no submission ops\n", vgpu->id);
> > + return 0;
> > + }
> > +
> > + ret = ops[interface]->init(vgpu);
> > + if (ret)
> > + return ret;
> > +
> > + s->ops = ops[interface];
> > + s->virtual_submission_interface = interface;
> > + s->active = true;
> > +
> > + gvt_dbg_core("vgpu%d: activate ops [ %s ]\n",
> > + vgpu->id, s->ops->name);
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > * intel_vgpu_destroy_workload - destroy a vGPU workload
> > * @vgpu: a vGPU
> > *
> > diff --git a/drivers/gpu/drm/i915/gvt/scheduler.h
> > b/drivers/gpu/drm/i915/gvt/scheduler.h
> > index b56b6de..5aa28fb 100644
> > --- a/drivers/gpu/drm/i915/gvt/scheduler.h
> > +++ b/drivers/gpu/drm/i915/gvt/scheduler.h
> > @@ -140,6 +140,12 @@ int intel_vgpu_setup_submission(struct
> > intel_vgpu *vgpu);
> >
> > void intel_vgpu_clean_submission(struct intel_vgpu *vgpu);
> >
> > +int intel_vgpu_select_submission_ops(struct intel_vgpu *vgpu,
> > + unsigned int interface);
> > +
> > +extern const struct intel_vgpu_submission_ops
> > +intel_vgpu_execlist_submission_ops;
> > +
> > struct intel_vgpu_workload *
> > intel_vgpu_create_workload(struct intel_vgpu *vgpu, int ring_id,
> > struct execlist_ctx_descriptor_format *desc); diff --git
> > a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
> > index 1c9818d..e63abd4 100644
> > --- a/drivers/gpu/drm/i915/gvt/vgpu.c
> > +++ b/drivers/gpu/drm/i915/gvt/vgpu.c
> > @@ -255,7 +255,6 @@ void intel_gvt_destroy_vgpu(struct intel_vgpu *vgpu)
> > idr_remove(&gvt->vgpu_idr, vgpu->id);
> > intel_vgpu_clean_sched_policy(vgpu);
> > intel_vgpu_clean_submission(vgpu);
> > - intel_vgpu_clean_execlist(vgpu);
> > intel_vgpu_clean_display(vgpu);
> > intel_vgpu_clean_opregion(vgpu);
> > intel_vgpu_clean_gtt(vgpu);
> > @@ -371,26 +370,20 @@ static struct intel_vgpu *__intel_gvt_create_vgpu(struct intel_gvt *gvt,
> > if (ret)
> > goto out_clean_gtt;
> >
> > - ret = intel_vgpu_init_execlist(vgpu);
> > - if (ret)
> > - goto out_clean_display;
> > -
> > ret = intel_vgpu_setup_submission(vgpu);
> > if (ret)
> > - goto out_clean_execlist;
> > + goto out_clean_display;
> >
> > ret = intel_vgpu_init_sched_policy(vgpu);
> > if (ret)
> > - goto out_clean_shadow_ctx;
> > + goto out_clean_submission;
> >
> > mutex_unlock(&gvt->lock);
> >
> > return vgpu;
> >
> > -out_clean_shadow_ctx:
> > +out_clean_submission:
> > intel_vgpu_clean_submission(vgpu);
> > -out_clean_execlist:
> > - intel_vgpu_clean_execlist(vgpu);
> > out_clean_display:
> > intel_vgpu_clean_display(vgpu);
> > out_clean_gtt:
> > --
> > 2.7.4
> >
> > _______________________________________________
> > intel-gvt-dev mailing list
> > intel-gvt-dev at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
>
> --
> Open Source Technology Center, Intel ltd.
>
> $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
> _______________________________________________
> intel-gvt-dev mailing list
> intel-gvt-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
--
Open Source Technology Center, Intel ltd.
$gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
More information about the intel-gvt-dev
mailing list