[Intel-xe] [PATCH 1/5] drm/xe: Introduce new selftests framework

[Lucas De Marchi]

On Mon, Mar 13, 2023 at 09:23:48AM -0700, Lucas De Marchi wrote:
>On Mon, Mar 13, 2023 at 09:13:12AM +0100, Michal Wajdeczko wrote:
>>Our initial attempt to run Kunit based tests on live devices is hard
>>to use and will not work well with multiple devices attached.
>>
>>Instead of relying on separate test module and duplicating support
>>for multiple live devices in every test entry point, create new set
>>of helpers and use test->priv to pass reference of the device under
>>test (DUT) directly to every test function.
>>
>>The same use pattern is supported for both mock and live selftests.
>>
>>By using bus_notifier we can start testing each new bound device
>>separatelly, subject to filtering using dut_filter modparam.
>>
>>All test cases can be included directly from .c file, allowing test
>>code to access and use any private functions.
>>
>>As Kunit will attempt to run all suites on module load, including
>>our new live test suites, we may want to allow to execute them on
>>any available device. This is controlled by anonymous_dut modparam.
>>
>>Support for either mock or live selftests, only one at time, is
>>controlled by new Kconfig options.
>>
>>Signed-off-by: Michal Wajdeczko <michal.wajdeczko at intel.com>
>>---
>>drivers/gpu/drm/xe/Kconfig.debug              |  22 ++++
>>drivers/gpu/drm/xe/Makefile                   |  11 ++
>>drivers/gpu/drm/xe/selftests/xe_selftests.h   |  35 ++++++
>>.../gpu/drm/xe/selftests/xe_selftests_auto.c  |  46 +++++++
>>.../drm/xe/selftests/xe_selftests_executor.c  | 118 ++++++++++++++++++
>>.../drm/xe/selftests/xe_selftests_helpers.c   | 101 +++++++++++++++
>>.../drm/xe/selftests/xe_selftests_internal.h  |  22 ++++
>>.../gpu/drm/xe/selftests/xe_selftests_mock.c  |  32 +++++
>>.../drm/xe/selftests/xe_selftests_params.c    |  23 ++++
>>drivers/gpu/drm/xe/xe_module.c                |   2 +
>>10 files changed, 412 insertions(+)
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests.h
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_auto.c
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_executor.c
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_helpers.c
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_internal.h
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_mock.c
>>create mode 100644 drivers/gpu/drm/xe/selftests/xe_selftests_params.c
>>
>>diff --git a/drivers/gpu/drm/xe/Kconfig.debug b/drivers/gpu/drm/xe/Kconfig.debug
>>index 93b284cdd0a2..aeba43977dca 100644
>>--- a/drivers/gpu/drm/xe/Kconfig.debug
>>+++ b/drivers/gpu/drm/xe/Kconfig.debug
>>@@ -61,6 +61,28 @@ config DRM_XE_SIMPLE_ERROR_CAPTURE
>>
>>	  If in doubt, say "N".
>>
>>+config DRM_XE_SELFTESTS_MOCK
>>+	bool "Mock selftests for Xe" if !KUNIT_ALL_TESTS
>>+	depends on DRM_XE && KUNIT
>>+	default KUNIT_ALL_TESTS
>>+	help
>>+		Choose this option to allow the driver to perform mock
>>+		selftests based on the Kunit framework.
>>+
>>+		Recommended for driver developers only.
>>+		If in doubt, say "N".
>>+
>>+config DRM_XE_SELFTESTS_LIVE
>>+	bool "Live selftests for Xe" if !DRM_XE_SELFTESTS_MOCK
>
>why?  I think we should aim to run live tests in *addition* to
>mock ones.  The mock tests are more testing our SW infra and can be used
>in a true "unit test" sense. While the live ones are testing the
>integration with the HW and making sure they work together.

on a similar note, I'd avoid grouping tests as "mock".  Any test
runnable under UML is a mock test and is testing the SW behavior.
 From a kunit perspective I think it's the live ones, interacting with
the real hardware that are the exception and can be grouped as "live"
tests.

Lucas De Marchi