[Intel-xe] [PATCH V9 6/6] drm/xe: Add min/max cap for engine scheduler properties
Matthew Brost
matthew.brost at intel.com
Fri Jul 28 14:24:02 UTC 2023
On Fri, Jul 28, 2023 at 01:36:29AM -0600, Upadhyay, Tejas wrote:
>
>
> > -----Original Message-----
> > From: Vishwanathapura, Niranjana <niranjana.vishwanathapura at intel.com>
> > Sent: Friday, July 28, 2023 11:56 AM
> > To: Upadhyay, Tejas <tejas.upadhyay at intel.com>
> > Cc: Brost, Matthew <matthew.brost at intel.com>; intel-
> > xe at lists.freedesktop.org
> > Subject: Re: [PATCH V9 6/6] drm/xe: Add min/max cap for engine scheduler
> > properties
> >
> > On Thu, Jul 27, 2023 at 10:54:42PM -0700, Upadhyay, Tejas wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Brost, Matthew <matthew.brost at intel.com>
> > >> Sent: Friday, July 28, 2023 10:35 AM
> > >> To: Vishwanathapura, Niranjana
> > <niranjana.vishwanathapura at intel.com>
> > >> Cc: Upadhyay, Tejas <tejas.upadhyay at intel.com>; intel-
> > >> xe at lists.freedesktop.org
> > >> Subject: Re: [PATCH V9 6/6] drm/xe: Add min/max cap for engine
> > >> scheduler properties
> > >>
> > >> On Thu, Jul 27, 2023 at 09:44:23PM -0700, Niranjana Vishwanathapura
> > >> wrote:
> > >> > On Thu, Jul 27, 2023 at 03:26:49PM +0000, Matthew Brost wrote:
> > >> > > On Thu, Jul 27, 2023 at 01:52:35PM +0530, Tejas Upadhyay wrote:
> > >> > > > Add sysfs entries for the min, max, and defaults for each of
> > >> > > > engine scheduler controls for every hardware engine class.
> > >> > > >
> > >> > > > Non-elevated user IOCTLs to set these controls must be within
> > >> > > > the min-max ranges of the sysfs entries, elevated user can set
> > >> > > > these controls to any value. However, introduced compile time
> > >> > > > CONFIG min-max values which restricts elevated user to be in
> > >> > > > compile time min-max range if at all sysfs min/max are violated.
> > >> > > >
> > >> > > > Sysfs entries examples are,
> > >> > > > DUT# cat
> > /sys/class/drm/cardX/device/tileN/gtN/engines/ccs/.defaults/
> > >> > > > job_timeout_max job_timeout_ms preempt_timeout_min
> > >> timeslice_duration_max timeslice_duration_us
> > >> > > > job_timeout_min preempt_timeout_max
> > preempt_timeout_us
> > >> timeslice_duration_min
> > >> > > >
> > >> > > > DUT# cat /sys/class/drm/card1/device/tileN/gtN/engines/ccs/
> > >> > > > .defaults/ job_timeout_min preempt_timeout_max
> > >> preempt_timeout_us timeslice_duration_min
> > >> > > > job_timeout_max job_timeout_ms preempt_timeout_min
> > >> timeslice_duration_max timeslice_duration_us
> > >> > > >
> > >> > > > V9 :
> > >> > > > - Rebase to use s/xe_engine/xe_hw_engine/ - Matt
> > >> > > > V8 :
> > >> > > > - fix enforce_sched_limit and avoid code duplication - Niranjana
> > >> > > > - Make sure min < max - Niranjana
> > >> > > > V7 :
> > >> > > > - Rebase to replace hw engine with eclass interface
> > >> > > > - return EINVAL in place of EPERM
> > >> > > > - Use some APIs to avoid code duplication
> > >> > > > V6 :
> > >> > > > - Rebase changes to reflect per engine class props interface - MattB
> > >> > > > - Use #if ENABLED - MattB
> > >> > > > - Remove MAX_SCHED_TIMEOUT check as range validation is
> > >> > > > enough
> > >> > > > V5 :
> > >> > > > - Rebase to resolve conflicts - CI
> > >> > > > V4 :
> > >> > > > - Rebase
> > >> > > > - Update commit to reflect tile addition
> > >> > > > - Use XE_HW macro directly as they are already filtered
> > >> > > > for CONFIG checks - Niranjana
> > >> > > > - Add CONFIG for enable/disable min/max limitation
> > >> > > > on elevated user. Default is enable - Matt/Joonas
> > >> > > > V3 :
> > >> > > > - Resolve CI hooks warning for kernel-doc
> > >> > > > V2 :
> > >> > > > - Restric min/max setting to #define default min/max for
> > >> > > > elevated user - Himal
> > >> > > > - Remove unrelated changes from patch - Niranjana
> > >> > > >
> > >> > > > Signed-off-by: Tejas Upadhyay <tejas.upadhyay at intel.com>
> > >> > > > ---
> > >> > > > drivers/gpu/drm/xe/Kconfig | 6 +
> > >> > > > drivers/gpu/drm/xe/Kconfig.profile | 46 +++
> > >> > > > drivers/gpu/drm/xe/xe_engine.c | 25 +-
> > >> > > > drivers/gpu/drm/xe/xe_hw_engine.c | 8 +
> > >> > > > drivers/gpu/drm/xe/xe_hw_engine.h | 31 ++
> > >> > > > drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c | 354
> > >> ++++++++++++++++++
> > >> > > > drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.h | 2 +
> > >> > > > 7 files changed, 466 insertions(+), 6 deletions(-) create
> > >> > > > mode
> > >> > > > 100644 drivers/gpu/drm/xe/Kconfig.profile
> > >> > > >
> > >> > > > diff --git a/drivers/gpu/drm/xe/Kconfig
> > >> > > > b/drivers/gpu/drm/xe/Kconfig index d44794f99338..0a4ea965645b
> > >> > > > 100644
> > >> > > > --- a/drivers/gpu/drm/xe/Kconfig
> > >> > > > +++ b/drivers/gpu/drm/xe/Kconfig
> > >> > > > @@ -83,3 +83,9 @@ depends on DRM_XE depends on EXPERT
> > source
> > >> > > > "drivers/gpu/drm/xe/Kconfig.debug"
> > >> > > > endmenu
> > >> > > > +
> > >> > > > +menu "drm/xe Profile Guided Optimisation"
> > >> > > > + visible if EXPERT
> > >> > > > + depends on DRM_XE
> > >> > > > + source "drivers/gpu/drm/xe/Kconfig.profile"
> > >> > > > +endmenu
> > >> > > > diff --git a/drivers/gpu/drm/xe/Kconfig.profile
> > >> > > > b/drivers/gpu/drm/xe/Kconfig.profile
> > >> > > > new file mode 100644
> > >> > > > index 000000000000..e72f15ec4bf6
> > >> > > > --- /dev/null
> > >> > > > +++ b/drivers/gpu/drm/xe/Kconfig.profile
> > >> > > > @@ -0,0 +1,46 @@
> > >> > > > +config DRM_XE_JOB_TIMEOUT_MAX
> > >> > > > + int "Default max job timeout (ms)"
> > >> > > > + default 10000 # milliseconds
> > >> > > > + help
> > >> > > > + Configures the default max job timeout after which job will
> > >> > > > + be forcefully taken away from scheduler.
> > >> > > > +config DRM_XE_JOB_TIMEOUT_MIN
> > >> > > > + int "Default max job timeout (ms)"
> > >> > > > + default 1 # milliseconds
> > >> > > > + help
> > >> > > > + Configures the default min job timeout after which job will
> > >> > > > + be forcefully taken away from scheduler.
> > >> > > > +config DRM_XE_TIMESLICE_MAX
> > >> > > > + int "Default max timeslice duration (us)"
> > >> > > > + default 10000000 # microseconds
> > >> > > > + help
> > >> > > > + Configures the default max timeslice duration between
> > multiple
> > >> > > > + contexts by guc scheduling.
> > >> > > > +config DRM_XE_TIMESLICE_MIN
> > >> > > > + int "Default min timeslice duration (us)"
> > >> > > > + default 1 # microseconds
> > >> > > > + help
> > >> > > > + Configures the default min timeslice duration between
> > multiple
> > >> > > > + contexts by guc scheduling.
> > >> > > > +config DRM_XE_PREEMPT_TIMEOUT_MAX
> > >> > > > + int "Default max preempt timeout (us)"
> > >> > > > + default 10000000 # microseconds
> > >> > > > + help
> > >> > > > + Configures the default max preempt timeout after which
> > context
> > >> > > > + will be forcefully taken away and higher priority context will
> > >> > > > + run.
> > >> > > > +config DRM_XE_PREEMPT_TIMEOUT_MIN
> > >> > > > + int "Default min preempt timeout (us)"
> > >> > > > + default 1 # microseconds
> > >> > > > + help
> > >> > > > + Configures the default min preempt timeout after which
> > context
> > >> > > > + will be forcefully taken away and higher priority context will
> > >> > > > + run.
> > >> > > > +config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT
> > >> > > > + bool "Default configuration of limitation on scheduler timeout"
> > >> > > > + default y
> > >> > > > + help
> > >> > > > + Configures the enablement of limitation on scheduler
> > timeout
> > >> > > > + to apply to applicable user. For elevated user, all above MIN
> > >> > > > + and MAX values will apply when this configuration is enable
> > to
> > >> > > > + apply limitation. By default limitation is applied.
> > >> > > > diff --git a/drivers/gpu/drm/xe/xe_engine.c
> > >> > > > b/drivers/gpu/drm/xe/xe_engine.c index
> > >> > > > 9e167b113963..d934196eb79f
> > >> > > > 100644
> > >> > > > --- a/drivers/gpu/drm/xe/xe_engine.c
> > >> > > > +++ b/drivers/gpu/drm/xe/xe_engine.c
> > >> > > > @@ -13,6 +13,7 @@
> > >> > > >
> > >> > > > #include "xe_device.h"
> > >> > > > #include "xe_gt.h"
> > >> > > > +#include "xe_hw_engine_class_sysfs.h"
> > >> > > > #include "xe_hw_fence.h"
> > >> > > > #include "xe_lrc.h"
> > >> > > > #include "xe_macros.h"
> > >> > > > @@ -191,8 +192,12 @@ static int engine_set_priority(struct
> > >> > > > xe_device *xe, struct xe_engine *e, static int
> > >> engine_set_timeslice(struct xe_device *xe, struct xe_engine *e,
> > >> > > > u64 value, bool create)
> > >> > > > {
> > >> > > > - if (!capable(CAP_SYS_NICE))
> > >> > > > - return -EPERM;
> > >> > > > + u32 min = e->hwe->eclass->sched_props.timeslice_min;
> > >> > > > + u32 max = e->hwe->eclass->sched_props.timeslice_max;
> > >> > >
> > >> > > So I screwed this up in my last comment, sorry.
> > >> > >
> > >> > > min = cap_nice ? KConfig option for min :
> > >> > > e->hwe->eclass->sched_props.timeslice_min;
> > >> > > same logic for max;
> > >> > >
> > >> > > Probably add helper for this.
> > >> > >
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(value, min, max))
> > >> > >
> > >> > > Then I think logic works too...
> > >> > >
> > >> > > if cap nice && !Kconfig option to enfore limits
> > >> > > (enforce_schedule_limit
> > >> returns false):
> > >> > > user can do whatever
> > >> > > else if cap nice
> > >> > > user within Kconfig range
> > >> > > else
> > >> > > user within sysfs range
> > >> > >
> > >> > > This is what we want in the end.
> > >> > >
> > >> >
> > >> > Matt, I think this is exactly what the previous revision of the
> > >> > patch was
> > >> doing.
> > >> >
> > >>
> > >> No... Previous version
> > >>
> > >> min = Kconfig option to enfore limits ? KConfig option for min :
> > >> e->hwe-
> > >> >eclass->sched_props.timeslice_min
> > >> same for max
> > >>
> > >> Results in:
> > >>
> > >> if cap nice && !Kconfig option to enfore limits
> > >> (enforce_schedule_limit returns false):
> > >> user can do whatever
> > >> else if cap_nice || Kconfig option to enfore limit
> > >> user within Kconfig range
> > >> else
> > >> user within sysfs range
> > >>
> > >> !cap_nice && Kconfig option to enfore limits == user within Kconfig
> > >> range
> > >> (previously) !cap_nice && Kconfig option to enfore limits == user
> > >> within sysfs range (my suggestion)
> > >
> > >In previous version we can achieve it by just adding one below check right?
> > >#if IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT) &&
> > cap_nice
> > >+ u32 min = XE_HW_ENGINE_TIMESLICE_MIN;
> > >+ u32 max = XE_HW_ENGINE_TIMESLICE_MAX; #else
> > >+ u32 min = e->hwe->eclass->sched_props.timeslice_min;
> > >+ u32 max = e->hwe->eclass->sched_props.timeslice_max;
> > >+#endif
> > >
> > >Please let me know!
> > >
> >
> > Ah I see. Yah, it is a limits selection issue.
> > We can do,
> >
> > u32 min = e->hwe->eclass->sched_props.timeslice_min;
> > u32 max = e->hwe->eclass->sched_props.timeslice_max;
> >
> > #if IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT)
> > if (cap_nice) {
> > u32 min = XE_HW_ENGINE_TIMESLICE_MIN;
> > u32 max = XE_HW_ENGINE_TIMESLICE_MAX;
> > }
> > #endif
> >
> > Matt, Tejas
> > Also, looks like you missed my another comment below.
> >
> > Niranjana
> >
> > >>
> > >> The logic is different and I believe we want what I'm suggesting.
> > >>
> > >> Matt
> > >>
> > >> > > > + return -EINVAL;
> > >> > > >
> > >> > > > return e->ops->set_timeslice(e, value); } @@ -201,8 +206,12
> > >> > > > @@ static int engine_set_preemption_timeout(struct xe_device *xe,
> > >> > > > struct xe_engine *e, u64
> > value,
> > >> > > > bool create)
> > >> > > > {
> > >> > > > - if (!capable(CAP_SYS_NICE))
> > >> > > > - return -EPERM;
> > >> > > > + u32 min = e->hwe->eclass-
> > >sched_props.preempt_timeout_min;
> > >> > > > + u32 max = e->hwe->eclass-
> > >sched_props.preempt_timeout_max;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(value, min, max))
> > >> > > > + return -EINVAL;
> > >> > > >
> > >> > > > return e->ops->set_preempt_timeout(e, value); } @@ -
> > 266,11
> > >> > > > +275,15 @@ static int engine_set_persistence(struct xe_device
> > >> > > > +*xe,
> > >> > > > struct xe_engine *e, static int engine_set_job_timeout(struct
> > >> > > > xe_device
> > >> *xe, struct xe_engine *e,
> > >> > > > u64 value, bool create)
> > >> > > > {
> > >> > > > + u32 min = e->hwe->eclass->sched_props.job_timeout_min;
> > >> > > > + u32 max = e->hwe->eclass->sched_props.job_timeout_max;
> > >> > > > +
> > >> > > > if (XE_IOCTL_DBG(xe, !create))
> > >> > > > return -EINVAL;
> > >> > > >
> > >> > > > - if (!capable(CAP_SYS_NICE))
> > >> > > > - return -EPERM;
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(value, min, max))
> > >> > > > + return -EINVAL;
> > >> > > >
> > >> > > > return e->ops->set_job_timeout(e, value); } diff --git
> > >> > > > a/drivers/gpu/drm/xe/xe_hw_engine.c
> > >> > > > b/drivers/gpu/drm/xe/xe_hw_engine.c
> > >> > > > index afa7d25c3852..e601bffe3b13 100644
> > >> > > > --- a/drivers/gpu/drm/xe/xe_hw_engine.c
> > >> > > > +++ b/drivers/gpu/drm/xe/xe_hw_engine.c
> > >> > > > @@ -364,8 +364,16 @@ static void hw_engine_init_early(struct
> > >> > > > xe_gt *gt, struct xe_hw_engine *hwe,
> > >> > > >
> > >> > > > if (!gt->eclass[hwe->class].sched_props.job_timeout_ms) {
> > >> > > > gt->eclass[hwe->class].sched_props.job_timeout_ms
> > = HZ *
> > >> 5;
> > >> > > > + gt->eclass[hwe-
> > >class].sched_props.job_timeout_min =
> > >> XE_HW_ENGINE_JOB_TIMEOUT_MIN;
> > >> > > > + gt->eclass[hwe-
> > >class].sched_props.job_timeout_max =
> > >> > > > +XE_HW_ENGINE_JOB_TIMEOUT_MAX;
> > >> > > > gt->eclass[hwe->class].sched_props.timeslice_us = 1
> > * 1000;
> > >> > > > + gt->eclass[hwe->class].sched_props.timeslice_min =
> > >> XE_HW_ENGINE_TIMESLICE_MIN;
> > >> > > > + gt->eclass[hwe->class].sched_props.timeslice_max =
> > >> > > > +XE_HW_ENGINE_TIMESLICE_MAX;
> > >> > > > gt->eclass[hwe-
> > >class].sched_props.preempt_timeout_us =
> > >> 640 *
> > >> > > > 1000;
> > >> > > > + gt->eclass[hwe-
> > >class].sched_props.preempt_timeout_min =
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN;
> > >> > > > + gt->eclass[hwe-
> > >class].sched_props.preempt_timeout_max =
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX;
> > >> > > > /* Record default props */
> > >> > > > gt->eclass[hwe->class].defaults = gt->eclass[hwe-
> > >> >class].sched_props;
> > >> > > > }
> > >> > > > diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h
> > >> > > > b/drivers/gpu/drm/xe/xe_hw_engine.h
> > >> > > > index 7eca9d53c7b1..3d37d6d44261 100644
> > >> > > > --- a/drivers/gpu/drm/xe/xe_hw_engine.h
> > >> > > > +++ b/drivers/gpu/drm/xe/xe_hw_engine.h
> > >> > > > @@ -10,6 +10,37 @@
> > >> > > >
> > >> > > > struct drm_printer;
> > >> > > >
> > >> > > > +#ifdef CONFIG_DRM_XE_JOB_TIMEOUT_MIN #define
> > >> > > > +XE_HW_ENGINE_JOB_TIMEOUT_MIN
> > >> CONFIG_DRM_XE_JOB_TIMEOUT_MIN #else
> > >> > > > +#define XE_HW_ENGINE_JOB_TIMEOUT_MIN 1 #endif #ifdef
> > >> > > > +CONFIG_DRM_XE_JOB_TIMEOUT_MAX #define
> > >> > > > +XE_HW_ENGINE_JOB_TIMEOUT_MAX
> > >> CONFIG_DRM_XE_JOB_TIMEOUT_MAX #else
> > >> > > > +#define XE_HW_ENGINE_JOB_TIMEOUT_MAX (10 * 1000) #endif
> > #ifdef
> > >> > > > +CONFIG_DRM_XE_TIMESLICE_MIN #define
> > >> XE_HW_ENGINE_TIMESLICE_MIN
> > >> > > > +CONFIG_DRM_XE_TIMESLICE_MIN #else #define
> > >> > > > +XE_HW_ENGINE_TIMESLICE_MIN 1 #endif #ifdef
> > >> > > > +CONFIG_DRM_XE_TIMESLICE_MAX #define
> > >> XE_HW_ENGINE_TIMESLICE_MAX
> > >> > > > +CONFIG_DRM_XE_TIMESLICE_MAX #else #define
> > >> > > > +XE_HW_ENGINE_TIMESLICE_MAX (10 * 1000 * 1000) #endif #ifdef
> > >> > > > +CONFIG_DRM_XE_PREEMPT_TIMEOUT_MIN
> > >> > > > +#define XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN
> > >> > > > +CONFIG_DRM_XE_PREEMPT_TIMEOUT_MIN #else #define
> > >> > > > +XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN 1 #endif #ifdef
> > >> > > > +CONFIG_DRM_XE_PREEMPT_TIMEOUT_MAX
> > >> > > > +#define XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX
> > >> > > > +CONFIG_DRM_XE_PREEMPT_TIMEOUT_MAX #else #define
> > >> > > > +XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX (10 * 1000 * 1000)
> > #endif
> > >> > > > +
> > >> > > > int xe_hw_engines_init_early(struct xe_gt *gt); int
> > >> > > > xe_hw_engines_init(struct xe_gt *gt); void
> > >> > > > xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16
> > >> > > > intr_vec); diff --git
> > >> > > > a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c
> > >> > > > b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c
> > >> > > > index 990bb675d1e0..2b7ac4e02db6 100644
> > >> > > > --- a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c
> > >> > > > +++ b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c
> > >> > > > @@ -11,6 +11,20 @@
> > >> > > >
> > >> > > > static int xe_add_hw_engine_class_defaults(struct kobject
> > >> > > > *parent);
> > >> > > >
> > >> > > > +bool enforce_schedule_limit(void) { #if
> > >> > > > +IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT)
> > >> > > > + return true;
> > >> > > > +#else
> > >> > > > + return !capable(CAP_SYS_NICE); #endif }
> > >> > > > +
> > >> > > > +bool engine_timeout_in_range(u64 timeout, u64 min, u64 max) {
> > >> > > > + return timeout >= min && timeout <= max; }
> > >> > > > +
> > >> > > > static void kobj_xe_hw_engine_release(struct kobject *kobj) {
> > >> > > > kfree(kobj);
> > >> > > > @@ -39,11 +53,92 @@ kobj_xe_hw_engine_class(struct kobject
> > >> *parent, char *name)
> > >> > > > return keclass;
> > >> > > > }
> > >> > > >
> > >> > > > +static ssize_t job_timeout_max_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t count) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 timeout;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &timeout);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (timeout < eclass->sched_props.job_timeout_min)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout,
> > >> > > > +
> > XE_HW_ENGINE_JOB_TIMEOUT_MIN,
> > >> > > > +
> > XE_HW_ENGINE_JOB_TIMEOUT_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.job_timeout_max,
> > timeout);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t job_timeout_max_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.job_timeout_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute job_timeout_max_attr =
> > >> > > > +__ATTR(job_timeout_max, 0644, job_timeout_max_show,
> > >> > > > +job_timeout_max_store);
> > >> > > > +
> > >> > > > +static ssize_t job_timeout_min_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t count) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 timeout;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &timeout);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (timeout > eclass->sched_props.job_timeout_max)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout,
> > >> > > > +
> > XE_HW_ENGINE_JOB_TIMEOUT_MIN,
> > >> > > > +
> > XE_HW_ENGINE_JOB_TIMEOUT_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.job_timeout_min,
> > timeout);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t job_timeout_min_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.job_timeout_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute job_timeout_min_attr =
> > >> > > > +__ATTR(job_timeout_min, 0644, job_timeout_min_show,
> > >> > > > +job_timeout_min_store);
> > >> > > > +
> > >> > > > static ssize_t job_timeout_store(struct kobject *kobj,
> > >> > > > struct kobj_attribute *attr,
> > >> > > > const char *buf, size_t count) {
> > >> > > > struct xe_hw_engine_class_intf *eclass =
> > >> > > > kobj_to_eclass(kobj);
> > >> > > > +#if IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT)
> > >> > > > + u32 min = XE_HW_ENGINE_JOB_TIMEOUT_MIN;
> > >> > > > + u32 max = XE_HW_ENGINE_JOB_TIMEOUT_MAX; #else
> > >> > > > + u32 min = e->hwe->eclass->sched_props.job_timeout_min;
> > >> > > > + u32 max = e->hwe->eclass->sched_props.job_timeout_max;
> > >> > > > +#endif
> > >> > >
> > >> > > I don't think we need this, just make sure default is within min
> > >> > > / max of the sched_props.
> > >> > >
> > >> >
> > >> > Shouldn't the sched_prop set through sysfs here which applies to
> > >> > all future user engines (xe_engine) has the same requirement as the
> > >> > sched_prop set directly to user engine (xe_engine) through
> > >> > set_properly
> > >> ioctl?
> > >> > ie., shoudln't the requirement be same in job_timeout_store() and
> > >> > engine_set_job_timeout()?
>
> I also think it should apply same to sysfs calls as well. Unless we think sysfs should limit to compile configs only. Since sysfs min/max cab be changed.
>
Disagree, the min / max sysfs entries are subject to the Kconfig limits.
So if you check the sysfs entry min / max here, the initial value will
always be within the Kconfig limits plus any additional restrictions a
user has imposed.
Matt
> Thanks,
> Tejas
>
> > >> >
> > >> > Niranjana
> > >> >
> > >> > > Matt
> > >> > >
> > >> > > > u32 timeout;
> > >> > > > int err;
> > >> > > >
> > >> > > > @@ -51,6 +146,10 @@ static ssize_t job_timeout_store(struct
> > >> > > > kobject
> > >> *kobj,
> > >> > > > if (err)
> > >> > > > return err;
> > >> > > >
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout, min, max))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > WRITE_ONCE(eclass->sched_props.job_timeout_ms,
> > timeout);
> > >> > > >
> > >> > > > return count;
> > >> > > > @@ -78,11 +177,40 @@ static ssize_t job_timeout_default(struct
> > >> > > > kobject *kobj, static struct kobj_attribute job_timeout_def =
> > >> > > > __ATTR(job_timeout_ms, 0444, job_timeout_default, NULL);
> > >> > > >
> > >> > > > +static ssize_t job_timeout_min_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->defaults.job_timeout_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute job_timeout_min_def =
> > >> > > > +__ATTR(job_timeout_min, 0444, job_timeout_min_default, NULL);
> > >> > > > +
> > >> > > > +static ssize_t job_timeout_max_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->defaults.job_timeout_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute job_timeout_max_def =
> > >> > > > +__ATTR(job_timeout_max, 0444, job_timeout_max_default, NULL);
> > >> > > > +
> > >> > > > static ssize_t timeslice_duration_store(struct kobject *kobj,
> > >> > > > struct kobj_attribute *attr,
> > >> > > > const char *buf, size_t count)
> > {
> > >> > > > struct xe_hw_engine_class_intf *eclass =
> > >> > > > kobj_to_eclass(kobj);
> > >> > > > +#if IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT)
> > >> > > > + u32 min = XE_HW_ENGINE_TIMESLICE_MIN;
> > >> > > > + u32 max = XE_HW_ENGINE_TIMESLICE_MAX; #else
> > >> > > > + u32 min = e->hwe->eclass->sched_props.timeslice_min;
> > >> > > > + u32 max = e->hwe->eclass->sched_props.timeslice_max;
> > >> > > > +#endif
> > >> > > > u32 duration;
> > >> > > > int err;
> > >> > > >
> > >> > > > @@ -90,11 +218,92 @@ static ssize_t
> > >> > > > timeslice_duration_store(struct
> > >> kobject *kobj,
> > >> > > > if (err)
> > >> > > > return err;
> > >> > > >
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(duration, min, max))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > WRITE_ONCE(eclass->sched_props.timeslice_us, duration);
> > >> > > >
> > >> > > > return count;
> > >> > > > }
> > >> > > >
> > >> > > > +static ssize_t timeslice_duration_max_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t
> > count) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 duration;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &duration);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (duration < eclass->sched_props.timeslice_min)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(duration,
> > >> > > > + XE_HW_ENGINE_TIMESLICE_MIN,
> > >> > > > +
> > XE_HW_ENGINE_TIMESLICE_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.timeslice_max, duration);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t timeslice_duration_max_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + char *buf)
> > >> > > > +{
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.timeslice_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute timeslice_duration_max_attr =
> > >> > > > + __ATTR(timeslice_duration_max, 0644,
> > >> timeslice_duration_max_show,
> > >> > > > + timeslice_duration_max_store);
> > >> > > > +
> > >> > > > +static ssize_t timeslice_duration_min_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t
> > count) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 duration;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &duration);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (duration > eclass->sched_props.timeslice_max)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(duration,
> > >> > > > + XE_HW_ENGINE_TIMESLICE_MIN,
> > >> > > > +
> > XE_HW_ENGINE_TIMESLICE_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.timeslice_min, duration);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t timeslice_duration_min_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > char
> > >> *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.timeslice_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute timeslice_duration_min_attr =
> > >> > > > + __ATTR(timeslice_duration_min, 0644,
> > >> timeslice_duration_min_show,
> > >> > > > + timeslice_duration_min_store);
> > >> > > > +
> > >> > > > static ssize_t timeslice_duration_show(struct kobject *kobj,
> > >> > > > struct kobj_attribute *attr, char *buf) { @@ -118,11
> > >> > > > +327,40 @@ static ssize_t timeslice_default(struct kobject
> > >> > > > *kobj, static struct kobj_attribute timeslice_duration_def =
> > >> > > > __ATTR(timeslice_duration_us, 0444, timeslice_default, NULL);
> > >> > > >
> > >> > > > +static ssize_t timeslice_min_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n", eclass->defaults.timeslice_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute timeslice_duration_min_def =
> > >> > > > +__ATTR(timeslice_duration_min, 0444, timeslice_min_default,
> > >> > > > +NULL);
> > >> > > > +
> > >> > > > +static ssize_t timeslice_max_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr, char
> > *buf) {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n", eclass->defaults.timeslice_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute timeslice_duration_max_def =
> > >> > > > +__ATTR(timeslice_duration_max, 0444, timeslice_max_default,
> > >> > > > +NULL);
> > >> > > > +
> > >> > > > static ssize_t preempt_timeout_store(struct kobject *kobj,
> > >> > > > struct kobj_attribute *attr,
> > >> > > > const char *buf, size_t count) {
> > >> > > > struct xe_hw_engine_class_intf *eclass =
> > >> > > > kobj_to_eclass(kobj);
> > >> > > > +#if IS_ENABLED(CONFIG_DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT)
> > >> > > > + u32 min = XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN;
> > >> > > > + u32 max = XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX; #else
> > >> > > > + u32 min = e->hwe->eclass-
> > >sched_props.preempt_timeout_min;
> > >> > > > + u32 max = e->hwe->eclass-
> > >sched_props.preempt_timeout_max;
> > >> > > > +#endif
> > >> > > > u32 timeout;
> > >> > > > int err;
> > >> > > >
> > >> > > > @@ -130,6 +368,10 @@ static ssize_t
> > >> > > > preempt_timeout_store(struct
> > >> kobject *kobj,
> > >> > > > if (err)
> > >> > > > return err;
> > >> > > >
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout, min, max))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > WRITE_ONCE(eclass->sched_props.preempt_timeout_us,
> > timeout);
> > >> > > >
> > >> > > > return count;
> > >> > > > @@ -158,17 +400,129 @@ static ssize_t
> > >> > > > preempt_timeout_default(struct kobject *kobj, static struct
> > >> > > > kobj_attribute preempt_timeout_def =
> > >> > > > __ATTR(preempt_timeout_us, 0444, preempt_timeout_default,
> > >> > > > NULL);
> > >> > > >
> > >> > > > +static ssize_t preempt_timeout_min_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + char *buf)
> > >> > > > +{
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->defaults.preempt_timeout_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute preempt_timeout_min_def =
> > >> > > > +__ATTR(preempt_timeout_min, 0444,
> > preempt_timeout_min_default,
> > >> > > > +NULL);
> > >> > > > +
> > >> > > > +static ssize_t preempt_timeout_max_default(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + char *buf)
> > >> > > > +{
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj->parent);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->defaults.preempt_timeout_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute preempt_timeout_max_def =
> > >> > > > +__ATTR(preempt_timeout_max, 0444,
> > >> preempt_timeout_max_default,
> > >> > > > +NULL);
> > >> > > > +
> > >> > > > +static ssize_t preempt_timeout_max_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t count)
> > {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 timeout;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &timeout);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (timeout < eclass->sched_props.preempt_timeout_min)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout,
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN,
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.preempt_timeout_max,
> > timeout);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t preempt_timeout_max_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > char *buf)
> > >> {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.preempt_timeout_max);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute preempt_timeout_max_attr =
> > >> > > > + __ATTR(preempt_timeout_max, 0644,
> > preempt_timeout_max_show,
> > >> > > > + preempt_timeout_max_store);
> > >> > > > +
> > >> > > > +static ssize_t preempt_timeout_min_store(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > >> > > > + const char *buf, size_t count)
> > {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > kobj_to_eclass(kobj);
> > >> > > > + u32 timeout;
> > >> > > > + int err;
> > >> > > > +
> > >> > > > + err = kstrtou32(buf, 0, &timeout);
> > >> > > > + if (err)
> > >> > > > + return err;
> > >> > > > +
> > >> > > > + if (timeout > eclass->sched_props.preempt_timeout_max)
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + if (enforce_schedule_limit() &&
> > >> > > > + !engine_timeout_in_range(timeout,
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MIN,
> > >> > > > +
> > >> XE_HW_ENGINE_PREEMPT_TIMEOUT_MAX))
> > >> > > > + return -EINVAL;
> > >> > > > +
> > >> > > > + WRITE_ONCE(eclass->sched_props.preempt_timeout_min,
> > timeout);
> > >> > > > +
> > >> > > > + return count;
> > >> > > > +}
> > >> > > > +
> > >> > > > +static ssize_t preempt_timeout_min_show(struct kobject *kobj,
> > >> > > > + struct kobj_attribute *attr,
> > char *buf)
> > >> {
> > >> > > > + struct xe_hw_engine_class_intf *eclass =
> > >> > > > +kobj_to_eclass(kobj);
> > >> > > > +
> > >> > > > + return sprintf(buf, "%u\n",
> > >> > > > +eclass->sched_props.preempt_timeout_min);
> > >> > > > +}
> > >> > > > +
> > >> > > > +static struct kobj_attribute preempt_timeout_min_attr =
> > >> > > > + __ATTR(preempt_timeout_min, 0644,
> > preempt_timeout_min_show,
> > >> > > > + preempt_timeout_min_store);
> > >> > > > +
> > >> > > > static const struct attribute *defaults[] = {
> > >> > > > &job_timeout_def.attr,
> > >> > > > + &job_timeout_min_def.attr,
> > >> > > > + &job_timeout_max_def.attr,
> > >> > > > ×lice_duration_def.attr,
> > >> > > > + ×lice_duration_min_def.attr,
> > >> > > > + ×lice_duration_max_def.attr,
> > >> > > > &preempt_timeout_def.attr,
> > >> > > > + &preempt_timeout_min_def.attr,
> > >> > > > + &preempt_timeout_max_def.attr,
> > >> > > > NULL
> > >> > > > };
> > >> > > >
> > >> > > > static const struct attribute *files[] = {
> > >> > > > &job_timeout_attr.attr,
> > >> > > > + &job_timeout_min_attr.attr,
> > >> > > > + &job_timeout_max_attr.attr,
> > >> > > > ×lice_duration_attr.attr,
> > >> > > > + ×lice_duration_min_attr.attr,
> > >> > > > + ×lice_duration_max_attr.attr,
> > >> > > > &preempt_timeout_attr.attr,
> > >> > > > + &preempt_timeout_min_attr.attr,
> > >> > > > + &preempt_timeout_max_attr.attr,
> > >> > > > NULL
> > >> > > > };
> > >> > > >
> > >> > > > diff --git a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.h
> > >> > > > b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.h
> > >> > > > index 757136614672..2e2ab351a991 100644
> > >> > > > --- a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.h
> > >> > > > +++ b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.h
> > >> > > > @@ -10,6 +10,8 @@
> > >> > > >
> > >> > > > #define MAX_ENGINE_CLASS_NAME_LEN 16
> > >> > > > int xe_hw_engine_class_sysfs_init(struct xe_gt *gt);
> > >> > > > +bool enforce_schedule_limit(void); bool
> > >> > > > +engine_timeout_in_range(u64 timeout, u64 min, u64 max);
> > >> > > >
> > >> > > > /**
> > >> > > > * struct kobj_eclass - A eclass's kobject struct that
> > >> > > > connects the kobject and the
> > >> > > > --
> > >> > > > 2.25.1
> > >> > > >
More information about the Intel-xe
mailing list