[PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
Lucas De Marchi
lucas.demarchi at intel.com
Fri Jun 28 15:55:06 UTC 2024
On Thu, Jun 20, 2024 at 12:52:05PM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>From: Aravind Iddamsetty <aravind.iddamsetty at linux.intel.com>
>>
>>There are a set of engine group busyness counters provided by HW which are
>>perfect fit to be exposed via PMU perf events.
>>
>>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>>events can be listed using:
>>perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>
>>and can be read using:
>>
>>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>> time counts unit events
>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>
>>The pmu base implementation is taken from i915.
>>
>>v2:
>>Store last known value when device is awake return that while the GT is
>>suspended and then update the driver copy when read during awake.
>>
>>v3:
>>1. drop init_samples, as storing counters before going to suspend should
>>be sufficient.
>>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>dropped helpers to store and read samples.
>>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>before reading the OA registers.
>>4. dropped format attr as no longer needed
>>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>6. few other nits.
>>
>>v4: minor nits.
>>
>>v5: take forcewake when accessing the OAG registers
>>
>>v6:
>>1. drop engine_busyness_sample_type
>>2. update UAPI documentation
>>
>>v7:
>>1. update UAPI documentation
>>2. drop MEDIA_GT specific change for media busyness counter.
>>
>>v8:
>>1. rebase
>>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>3. remove interrupts pmu event
>>
>>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>
>>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
>>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
>>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu at intel.com>
>>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu at intel.com>
>>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty at linux.intel.com>
>>Reviewed-by: Ashutosh Dixit <ashutosh.dixit at intel.com>
>>Signed-off-by: Riana Tauro <riana.tauro at intel.com>
>>---
>>drivers/gpu/drm/xe/Makefile | 2 +
>>drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>drivers/gpu/drm/xe/xe_device.c | 2 +
>>drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>drivers/gpu/drm/xe/xe_gt.c | 2 +
>>drivers/gpu/drm/xe/xe_module.c | 5 +
>>drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>include/uapi/drm/xe_drm.h | 39 ++
>>10 files changed, 783 insertions(+)
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>index cbf961b90237..83bf1e07669b 100644
>>--- a/drivers/gpu/drm/xe/Makefile
>>+++ b/drivers/gpu/drm/xe/Makefile
>>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/skl_universal_plane.o \
>> i915-display/skl_watermark.o
>>
>>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>+
>>ifeq ($(CONFIG_ACPI),y)
>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/intel_acpi.o \
>>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>index 47c26c37608d..22821dcd4e1b 100644
>>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>@@ -390,6 +390,11 @@
>>#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>
>>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>+
>>#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>
>>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>index 64691a56d59c..bb00c8c9ec9b 100644
>>--- a/drivers/gpu/drm/xe/xe_device.c
>>+++ b/drivers/gpu/drm/xe/xe_device.c
>>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>
>> xe_hwmon_register(xe);
>>
>>+ xe_pmu_register(&xe->pmu);
>>+
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>
>>err_fini_display:
>>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>index 52bc461171d5..a5dba7325cf1 100644
>>--- a/drivers/gpu/drm/xe/xe_device_types.h
>>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>>@@ -18,6 +18,7 @@
>>#include "xe_lmtt_types.h"
>>#include "xe_memirq_types.h"
>>#include "xe_platform_types.h"
>>+#include "xe_pmu.h"
>>#include "xe_pt_types.h"
>>#include "xe_sriov_types.h"
>>#include "xe_step_types.h"
>>@@ -473,6 +474,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>>
>>+ /** @pmu: performance monitoring unit */
>>+ struct xe_pmu pmu;
>>+
>> /* private: */
>>
>>#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>index 57d84751e160..477d0ae5f230 100644
>>--- a/drivers/gpu/drm/xe/xe_gt.c
>>+++ b/drivers/gpu/drm/xe/xe_gt.c
>>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>> if (err)
>> goto err_msg;
>>
>>+ xe_pmu_suspend(gt);
>>+
>> err = xe_uc_suspend(>->uc);
>> if (err)
>> goto err_force_wake;
>>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>index 3edeb30d5ccb..26f814f97fc2 100644
>>--- a/drivers/gpu/drm/xe/xe_module.c
>>+++ b/drivers/gpu/drm/xe/xe_module.c
>>@@ -11,6 +11,7 @@
>>#include "xe_drv.h"
>>#include "xe_hw_fence.h"
>>#include "xe_pci.h"
>>+#include "xe_pmu.h"
>>#include "xe_sched_job.h"
>>
>>struct xe_modparam xe_modparam = {
>>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>>+ {
>>+ .init = xe_pmu_init,
>>+ .exit = xe_pmu_exit,
>>+ },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>new file mode 100644
>>index 000000000000..64960a358af2
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>>@@ -0,0 +1,631 @@
>>+// SPDX-License-Identifier: MIT
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#include <drm/drm_drv.h>
>>+#include <drm/drm_managed.h>
>>+#include <drm/xe_drm.h>
>>+
>>+#include "regs/xe_gt_regs.h"
>>+#include "xe_device.h"
>>+#include "xe_force_wake.h"
>>+#include "xe_gt_clock.h"
>>+#include "xe_mmio.h"
>>+#include "xe_macros.h"
>>+#include "xe_pm.h"
>>+
>>+static cpumask_t xe_pmu_cpumask;
>>+static unsigned int xe_pmu_target_cpu = -1;
>>+
>>+static unsigned int config_gt_id(const u64 config)
>>+{
>>+ return config >> __XE_PMU_GT_SHIFT;
>>+}
>>+
>>+static u64 config_counter(const u64 config)
>>+{
>>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>+}
>>+
>>+static void xe_pmu_event_destroy(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+
>>+ drm_WARN_ON(&xe->drm, event->parent);
>>+
>>+ drm_dev_put(&xe->drm);
>>+}
>>+
>>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>+{
>>+ u64 val;
>>+
>>+ switch (sample_type) {
>>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>+ break;
>>+ default:
>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>+ }
>>+
>>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>+}
>>+
>>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>+{
>>+ int sample_type = config_counter(config);
>>+ const unsigned int gt_id = gt->info.id;
>>+ struct xe_device *xe = gt->tile->xe;
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ unsigned long flags;
>>+ bool device_awake;
>>+ u64 val;
>>+
>>+ device_awake = xe_pm_runtime_get_if_active(xe);
>>+ if (device_awake) {
>>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>+ val = __engine_group_busyness_read(gt, sample_type);
>>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>+ xe_pm_runtime_put(xe);
>>+ }
>>+
>>+ spin_lock_irqsave(&pmu->lock, flags);
>>+
>>+ if (device_awake)
>>+ pmu->sample[gt_id][sample_type] = val;
>>+ else
>>+ val = pmu->sample[gt_id][sample_type];
>>+
>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>+
>>+ return val;
>>+}
>>+
>>+static void engine_group_busyness_store(struct xe_gt *gt)
>>+{
>>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>>+ unsigned int gt_id = gt->info.id;
>>+ unsigned long flags;
>>+ int i;
>>+
>>+ spin_lock_irqsave(&pmu->lock, flags);
>>+
>>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>+
>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>+}
>>+
>>+static int
>>+config_status(struct xe_device *xe, u64 config)
>>+{
>>+ unsigned int gt_id = config_gt_id(config);
>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>+
>>+ if (gt_id >= XE_PMU_MAX_GT)
>>+ return -ENOENT;
>>+
>>+ switch (config_counter(config)) {
>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>>+ return -ENOENT;
>>+ break;
>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>+ return -ENOENT;
>>+ break;
>>+ default:
>>+ return -ENOENT;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static int xe_pmu_event_init(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ int ret;
>>+
>>+ if (pmu->closed)
>>+ return -ENODEV;
>>+
>>+ if (event->attr.type != event->pmu->type)
>>+ return -ENOENT;
>>+
>>+ /* unsupported modes and filters */
>>+ if (event->attr.sample_period) /* no sampling */
>>+ return -EINVAL;
>>+
>>+ if (has_branch_stack(event))
>>+ return -EOPNOTSUPP;
>>+
>>+ if (event->cpu < 0)
>>+ return -EINVAL;
>>+
>>+ /* only allow running on one cpu at a time */
>>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>+ return -EINVAL;
>>+
>>+ ret = config_status(xe, event->attr.config);
>>+ if (ret)
>>+ return ret;
>>+
>>+ if (!event->parent) {
>>+ drm_dev_get(&xe->drm);
>>+ event->destroy = xe_pmu_event_destroy;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static u64 __xe_pmu_event_read(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>>+ const u64 config = event->attr.config;
>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>+ u64 val;
>>+
>>+ switch (config_counter(config)) {
>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>+ val = engine_group_busyness_read(gt, config);
>>+ break;
>>+ default:
>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>+ }
>>+
>>+ return val;
>>+}
>>+
>>+static void xe_pmu_event_read(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct hw_perf_event *hwc = &event->hw;
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ u64 prev, new;
>>+
>>+ if (pmu->closed) {
>>+ event->hw.state = PERF_HES_STOPPED;
>>+ return;
>>+ }
>>+again:
>>+ prev = local64_read(&hwc->prev_count);
>>+ new = __xe_pmu_event_read(event);
>>+
>>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>+ goto again;
>>+
>>+ local64_add(new - prev, &event->count);
>>+}
>>+
>>+static void xe_pmu_enable(struct perf_event *event)
>>+{
>>+ /*
>>+ * Store the current counter value so we can report the correct delta
>>+ * for all listeners. Even when the event was already enabled and has
>>+ * an existing non-zero value.
>>+ */
>>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>+}
>>+
>>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+
>>+ if (pmu->closed)
>>+ return;
>>+
>>+ xe_pmu_enable(event);
>>+ event->hw.state = 0;
>>+}
>>+
>>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>+{
>>+ if (flags & PERF_EF_UPDATE)
>>+ xe_pmu_event_read(event);
>>+
>>+ event->hw.state = PERF_HES_STOPPED;
>>+}
>>+
>>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+
>>+ if (pmu->closed)
>>+ return -ENODEV;
>>+
>>+ if (flags & PERF_EF_START)
>>+ xe_pmu_event_start(event, flags);
>>+
>>+ return 0;
>>+}
>>+
>>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>>+{
>>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>+}
>>+
>>+static int xe_pmu_event_event_idx(struct perf_event *event)
>>+{
>>+ return 0;
>>+}
>>+
>>+struct xe_ext_attribute {
>>+ struct device_attribute attr;
>>+ unsigned long val;
>>+};
>>+
>>+static ssize_t xe_pmu_event_show(struct device *dev,
>>+ struct device_attribute *attr, char *buf)
>>+{
>>+ struct xe_ext_attribute *eattr;
>>+
>>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>>+}
>>+
>>+static ssize_t cpumask_show(struct device *dev,
>>+ struct device_attribute *attr, char *buf)
>>+{
>>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>+}
>>+
>>+static DEVICE_ATTR_RO(cpumask);
>>+
>>+static struct attribute *xe_cpumask_attrs[] = {
>>+ &dev_attr_cpumask.attr,
>>+ NULL,
>>+};
>>+
>>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>+ .attrs = xe_cpumask_attrs,
>>+};
>>+
>>+#define __event(__counter, __name, __unit) \
>>+{ \
>>+ .counter = (__counter), \
>>+ .name = (__name), \
>>+ .unit = (__unit), \
>>+}
>>+
>>+static struct xe_ext_attribute *
>>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>+{
>>+ sysfs_attr_init(&attr->attr.attr);
>>+ attr->attr.attr.name = name;
>>+ attr->attr.attr.mode = 0444;
>>+ attr->attr.show = xe_pmu_event_show;
>>+ attr->val = config;
>>+
>>+ return ++attr;
>>+}
>>+
>>+static struct perf_pmu_events_attr *
>>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>+ const char *str)
>>+{
>>+ sysfs_attr_init(&attr->attr.attr);
>>+ attr->attr.attr.name = name;
>>+ attr->attr.attr.mode = 0444;
>>+ attr->attr.show = perf_event_sysfs_show;
>>+ attr->event_str = str;
>>+
>>+ return ++attr;
>>+}
>>+
>>+static struct attribute **
>>+create_event_attributes(struct xe_pmu *pmu)
>>+{
>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>+ static const struct {
>>+ unsigned int counter;
>>+ const char *name;
>>+ const char *unit;
>>+ } events[] = {
>>+ __event(0, "render-group-busy", "ns"),
>>+ __event(1, "copy-group-busy", "ns"),
>>+ __event(2, "media-group-busy", "ns"),
>>+ __event(3, "any-engine-group-busy", "ns"),
>>+ };
>>+
>>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>+ struct attribute **attr = NULL, **attr_iter;
>>+ unsigned int count = 0;
>>+ unsigned int i, j;
>>+ struct xe_gt *gt;
>>+
>>+ /* Count how many counters we will be exposing. */
>>+ for_each_gt(gt, xe, j) {
>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>+
>>+ if (!config_status(xe, config))
>>+ count++;
>>+ }
>>+ }
>>+
>>+ /* Allocate attribute objects and table. */
>>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>+ if (!xe_attr)
>>+ goto err_alloc;
>>+
>>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>+ if (!pmu_attr)
>>+ goto err_alloc;
>>+
>>+ /* Max one pointer of each attribute type plus a termination entry. */
>>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>+ if (!attr)
>>+ goto err_alloc;
>>+
>>+ xe_iter = xe_attr;
>>+ pmu_iter = pmu_attr;
>>+ attr_iter = attr;
>>+
>>+ for_each_gt(gt, xe, j) {
>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>+ char *str;
>>+
>>+ if (config_status(xe, config))
>>+ continue;
>>+
>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>+ events[i].name, j);
>>+ if (!str)
>>+ goto err;
>>+
>>+ *attr_iter++ = &xe_iter->attr.attr;
>>+ xe_iter = add_xe_attr(xe_iter, str, config);
>>+
>>+ if (events[i].unit) {
>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>+ events[i].name, j);
>>+ if (!str)
>>+ goto err;
>>+
>>+ *attr_iter++ = &pmu_iter->attr.attr;
>>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>>+ events[i].unit);
>>+ }
>>+ }
>>+ }
>>+
>>+ pmu->xe_attr = xe_attr;
>>+ pmu->pmu_attr = pmu_attr;
>>+
>>+ return attr;
>>+
>>+err:
>>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>>+ kfree((*attr_iter)->name);
>>+
>>+err_alloc:
>>+ kfree(attr);
>>+ kfree(xe_attr);
>>+ kfree(pmu_attr);
>>+
>>+ return NULL;
>>+}
>>+
>>+static void free_event_attributes(struct xe_pmu *pmu)
>>+{
>>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>+
>>+ for (; *attr_iter; attr_iter++)
>>+ kfree((*attr_iter)->name);
>>+
>>+ kfree(pmu->events_attr_group.attrs);
>>+ kfree(pmu->xe_attr);
>>+ kfree(pmu->pmu_attr);
>>+
>>+ pmu->events_attr_group.attrs = NULL;
>>+ pmu->xe_attr = NULL;
>>+ pmu->pmu_attr = NULL;
>>+}
>>+
>>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>+{
>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>+
>>+ /* Select the first online CPU as a designated reader. */
>>+ if (cpumask_empty(&xe_pmu_cpumask))
>>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>+
>>+ return 0;
>>+}
>>+
>>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>+{
>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>+ unsigned int target = xe_pmu_target_cpu;
>>+
>>+ /*
>>+ * Unregistering an instance generates a CPU offline event which we must
>>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>+ */
>>+ if (pmu->closed)
>>+ return 0;
>>+
>>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>+
>>+ /* Migrate events if there is a valid target */
>>+ if (target < nr_cpu_ids) {
>>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>>+ xe_pmu_target_cpu = target;
>>+ }
>>+ }
>>+
>>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>>+ pmu->cpuhp.cpu = target;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>+
>>+int xe_pmu_init(void)
>>+{
>>+ int ret;
>>+
>>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>+ "perf/x86/intel/xe:online",
>>+ xe_pmu_cpu_online,
>>+ xe_pmu_cpu_offline);
>>+ if (ret < 0)
>>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>+ ret);
>>+ else
>>+ cpuhp_slot = ret;
>>+
>>+ return 0;
>>+}
>>+
>>+void xe_pmu_exit(void)
>>+{
>>+ if (cpuhp_slot != CPUHP_INVALID)
>>+ cpuhp_remove_multi_state(cpuhp_slot);
>>+}
>>+
>>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>+{
>>+ if (cpuhp_slot == CPUHP_INVALID)
>>+ return -EINVAL;
>>+
>>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>+}
>>+
>>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>+{
>>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>+}
>>+
>>+void xe_pmu_suspend(struct xe_gt *gt)
>>+{
>>+ engine_group_busyness_store(gt);
>>+}
>>+
>>+static void xe_pmu_unregister(void *arg)
>>+{
>>+ struct xe_pmu *pmu = arg;
>>+
>>+ if (!pmu->base.event_init)
>>+ return;
>>+
>>+ /*
>>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>+ * ensures all currently executing ones will have exited before we
>>+ * proceed with unregistration.
>>+ */
>>+ pmu->closed = true;
>>+ synchronize_rcu();
>>+
>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>+
>>+ perf_pmu_unregister(&pmu->base);
>>+ pmu->base.event_init = NULL;
>>+ kfree(pmu->base.attr_groups);
>>+ kfree(pmu->name);
>>+ free_event_attributes(pmu);
>>+}
>>+
>>+void xe_pmu_register(struct xe_pmu *pmu)
>>+{
>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>+ const struct attribute_group *attr_groups[] = {
>>+ &pmu->events_attr_group,
>>+ &xe_pmu_cpumask_attr_group,
>>+ NULL
>>+ };
>>+
>>+ int ret = -ENOMEM;
>>+
>>+ spin_lock_init(&pmu->lock);
>>+ pmu->cpuhp.cpu = -1;
>>+
>>+ pmu->name = kasprintf(GFP_KERNEL,
>>+ "xe_%s",
>>+ dev_name(xe->drm.dev));
>>+ if (pmu->name)
>>+ /* tools/perf reserves colons as special. */
>>+ strreplace((char *)pmu->name, ':', '_');
>>+
>>+ if (!pmu->name)
>>+ goto err;
>>+
>>+ pmu->events_attr_group.name = "events";
>>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>+ if (!pmu->events_attr_group.attrs)
>>+ goto err_name;
>>+
>>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>+ GFP_KERNEL);
>>+ if (!pmu->base.attr_groups)
>>+ goto err_attr;
>>+
>>+ pmu->base.module = THIS_MODULE;
>>+ pmu->base.task_ctx_nr = perf_invalid_context;
>>+ pmu->base.event_init = xe_pmu_event_init;
>>+ pmu->base.add = xe_pmu_event_add;
>>+ pmu->base.del = xe_pmu_event_del;
>>+ pmu->base.start = xe_pmu_event_start;
>>+ pmu->base.stop = xe_pmu_event_stop;
>>+ pmu->base.read = xe_pmu_event_read;
>>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>>+
>>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>+ if (ret)
>>+ goto err_groups;
>>+
>>+ ret = xe_pmu_register_cpuhp_state(pmu);
>>+ if (ret)
>>+ goto err_unreg;
>>+
>>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>+ if (ret)
>>+ goto err_cpuhp;
>>+
>>+ return;
>>+
>>+err_cpuhp:
>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>+err_unreg:
>>+ perf_pmu_unregister(&pmu->base);
>>+err_groups:
>>+ kfree(pmu->base.attr_groups);
>>+err_attr:
>>+ pmu->base.event_init = NULL;
>>+ free_event_attributes(pmu);
>>+err_name:
>>+ kfree(pmu->name);
>>+err:
>>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>>+}
>>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>new file mode 100644
>>index 000000000000..8afa256f9dac
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>>@@ -0,0 +1,26 @@
>>+/* SPDX-License-Identifier: MIT */
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#ifndef _XE_PMU_H_
>>+#define _XE_PMU_H_
>>+
>>+#include "xe_pmu_types.h"
>>+
>>+struct xe_gt;
>>+
>>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>+int xe_pmu_init(void);
>>+void xe_pmu_exit(void);
>>+void xe_pmu_register(struct xe_pmu *pmu);
>>+void xe_pmu_suspend(struct xe_gt *gt);
>>+#else
>>+static inline int xe_pmu_init(void) { return 0; }
>>+static inline void xe_pmu_exit(void) {}
>>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>+#endif
>>+
>>+#endif
>>+
>>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>new file mode 100644
>>index 000000000000..e86e8d7e0356
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>@@ -0,0 +1,67 @@
>>+/* SPDX-License-Identifier: MIT */
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#ifndef _XE_PMU_TYPES_H_
>>+#define _XE_PMU_TYPES_H_
>>+
>>+#include <linux/perf_event.h>
>>+#include <linux/spinlock_types.h>
>>+#include <uapi/drm/xe_drm.h>
>>+
>>+enum {
>>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>+ __XE_NUM_PMU_SAMPLERS
>>+};
>>+
>>+#define XE_PMU_MAX_GT 2
>>+
>>+struct xe_pmu {
>>+ /**
>>+ * @cpuhp: Struct used for CPU hotplug handling.
>>+ */
>>+ struct {
>>+ struct hlist_node node;
>>+ unsigned int cpu;
>>+ } cpuhp;
>>+ /**
>>+ * @base: PMU base.
>>+ */
>>+ struct pmu base;
>>+ /**
>>+ * @closed: xe is unregistering.
>>+ */
>>+ bool closed;
>>+ /**
>>+ * @name: Name as registered with perf core.
>>+ */
>>+ const char *name;
>>+ /**
>>+ * @lock: Lock protecting enable mask and ref count handling.
>>+ */
>>+ spinlock_t lock;
>>+ /**
>>+ * @sample: Current and previous (raw) counters.
>>+ *
>>+ * These counters are updated when the device is awake.
>>+ */
>>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>+ /**
>>+ * @events_attr_group: Device events attribute group.
>>+ */
>>+ struct attribute_group events_attr_group;
>>+ /**
>>+ * @xe_attr: Memory block holding device attributes.
>>+ */
>>+ void *xe_attr;
>>+ /**
>>+ * @pmu_attr: Memory block holding device attributes.
>>+ */
>>+ void *pmu_attr;
>>+};
>>+
>>+#endif
>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>index d7b0903c22b2..07ca545354f7 100644
>>--- a/include/uapi/drm/xe_drm.h
>>+++ b/include/uapi/drm/xe_drm.h
>>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>>};
>>
>>+/**
>>+ * DOC: XE PMU event config IDs
>>+ *
>>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>+ * particular event.
>>+ *
>>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>+ *
>>+ * .. code-block:: C
>>+ *
>>+ * struct perf_event_attr attr;
>>+ * long long count;
>>+ * int cpu = 0;
>>+ * int fd;
>>+ *
>>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>+ * attr.use_clockid = 1;
>>+ * attr.clockid = CLOCK_MONOTONIC;
>>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>+ *
>>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>+ */
>>+
>>+/*
>>+ * Top bits of every counter are GT id.
>>+ */
>>+#define __XE_PMU_GT_SHIFT (56)
>>+
>>+#define ___XE_PMU_OTHER(gt, x) \
>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>+
>>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>
>+ Lucas for inputs
>
>We should align this to the interface planned for other PMU busyness
>counters as well as how we do PCEU. i.e.
>
>1) counters are in ticks
>2) total time in ticks is also exported to the user.
>
>For 1), I would just append TICKS to the counter names and drop the
this uses perf and as such I believe we should use the terms used by
perf.
$ sudo perf stat sleep 1
Performance counter stats for 'sleep 1':
0.91 msec task-clock # 0.001 CPUs utilized
1 context-switches # 1.096 K/sec
0 cpu-migrations # 0.000 /sec
72 page-faults # 78.924 K/sec
------> 2,033,156 cycles # 2.229 GHz
1,560,992 instructions # 0.77 insn per cycle
290,814 branches # 318.779 M/sec
10,449 branch-misses # 3.59% of all branches
1.001580466 seconds time elapsed
0.000000000 seconds user
0.001545000 seconds sys
so... s/ticks/cycles/
I think I said that before, but what's up with all these "group" in the
names? It's confusing since apparently group and engine class are mixed.
We are also missing proper kernel-doc in xe_pmu.c
Lucas De Marchi
>conversion to _ns in __engine_group_busyness_read(). Also, drop the
>patch that adds this conversion helper.
>
>For 2) define a new counter - total active ticks that would return the
>'CPU' timestamp converted to gpu ticks. The reason I am insisting on
>CPU timestamp here is because we want to have a time base that is
>ticking even when the GPU is idle.
>
>Regards,
>Umesh
>
>>+
>>#if defined(__cplusplus)
>>}
>>#endif
>>--
>>2.40.0
>>
More information about the Intel-xe
mailing list