dri-devel August 2021

dri-devel@lists.freedesktop.org

324 participants
762 discussions

[PATCH v1 00/14] Add MEMORY_DEVICE_PUBLIC for CPU-accessible coherent device memory
by Alex Sierra 27 Aug '21

27 Aug '21

AMD is building a system architecture for the Frontier supercomputer with a coherent interconnect between CPUs and GPUs. This hardware architecture allows the CPUs to coherently access GPU device memory. We have hardware in our labs and we are working with our partner HPE on the BIOS, firmware and software for delivery to the DOE. The system BIOS advertises the GPU device memory (aka VRAM) as SPM (special purpose memory) in the UEFI system address map. The amdgpu driver registers the memory with devmap as MEMORY_DEVICE_PUBLIC using devm_memremap_pages. This patch series adds MEMORY_DEVICE_PUBLIC, which is similar to MEMORY_DEVICE_GENERIC in that it can be mapped for CPU access, but adds support for migrating this memory similar to MEMORY_DEVICE_PRIVATE. We also included and updated two patches from Ralph Campbell (Nvidia), which change ZONE_DEVICE reference counting as requested in previous reviews of this patch series (see https://patchwork.freedesktop.org/series/90706/). Finally, we extended hmm_test to cover migration of MEMORY_DEVICE_PUBLIC. This work is based on HMM and our SVM memory manager, which has landed in Linux 5.14 recently. Alex Sierra (12): mm: add iomem vma selection for memory migration mm: add zone device public type memory support drm/amdkfd: ref count init for device pages drm/amdkfd: add SPM support for SVM drm/amdkfd: public type as sys mem on migration to ram mm: add public type support to migrate_vma helpers mm: call pgmap->ops->page_free for DEVICE_PUBLIC pages lib: test_hmm add ioctl to get zone device type lib: test_hmm add module param for zone device type lib: add support for device public type in test_hmm tools: update hmm-test to support device public type tools: update test_hmm script to support SP config Ralph Campbell (2): ext4/xfs: add page refcount helper mm: remove extra ZONE_DEVICE struct page refcount arch/powerpc/kvm/book3s_hv_uvmem.c | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 36 ++-- drivers/gpu/drm/nouveau/nouveau_dmem.c | 2 +- fs/dax.c | 8 +- fs/ext4/inode.c | 5 +- fs/fuse/dax.c | 4 +- fs/xfs/xfs_file.c | 4 +- include/linux/dax.h | 10 + include/linux/memremap.h | 15 +- include/linux/migrate.h | 1 + include/linux/mm.h | 19 +- lib/test_hmm.c | 247 +++++++++++++++-------- lib/test_hmm_uapi.h | 16 ++ mm/internal.h | 8 + mm/memcontrol.c | 6 +- mm/memory-failure.c | 6 +- mm/memremap.c | 70 ++----- mm/migrate.c | 27 +-- mm/page_alloc.c | 3 + mm/swap.c | 45 +---- tools/testing/selftests/vm/hmm-tests.c | 142 +++++++++++-- tools/testing/selftests/vm/test_hmm.sh | 20 +- 22 files changed, 443 insertions(+), 253 deletions(-) -- 2.32.0

6 22

[RFC PATCH 4/4] drm: Make fbdev emulation depend on FB_CORE instead of FB
by Javier Martinez Canillas 27 Aug '21

27 Aug '21

Now that the fbdev core has been split in FB_CORE and FB, make DRM fbdev emulation layer to just depend on the former. This allows to disable the CONFIG_FB option if is not needed, which will avoid the need to explicitly disable all the legacy fbdev drivers. Signed-off-by: Javier Martinez Canillas <javierm(a)redhat.com> --- drivers/gpu/drm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index 8fc40317f2b..7806adb02f1 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -103,7 +103,7 @@ config DRM_DEBUG_DP_MST_TOPOLOGY_REFS config DRM_FBDEV_EMULATION bool "Enable legacy fbdev support for your modesetting driver" depends on DRM - depends on FB + depends on FB_CORE select DRM_KMS_HELPER select FB_CFB_FILLRECT select FB_CFB_COPYAREA -- 2.31.1

1 0

[RFC PATCH 3/4] fbdev: Split frame buffer support in FB and FB_CORE symbols
by Javier Martinez Canillas 27 Aug '21

27 Aug '21

Currently the CONFIG_FB option has to be enabled even if no legacy fbdev drivers are needed, in order to have support for a framebuffer console. The DRM subsystem has a fbdev emulation layer, but depends on CONFIG_FB and so it can only be enabled if that dependency is enabled as well. That means fbdev drivers have to be explicitly disabled if users want to enable CONFIG_FB, only to use fbcon with the DRM fbdev emulation layer. This patch introduces a CONFIG_FB_CORE option that could be enabled just to have the core support needed for CONFIG_DRM_FBDEV_EMULATION, allowing CONFIG_FB to be disabled (and automatically disabling all fbdev drivers). The fbsysfs.o object is left out of fb_core.o, because that's not needed for KMS drivers and the DRM fbdev emulation layer. Signed-off-by: Javier Martinez Canillas <javierm(a)redhat.com> --- arch/x86/Makefile | 2 +- arch/x86/video/Makefile | 2 +- drivers/video/console/Kconfig | 2 +- drivers/video/fbdev/Kconfig | 57 ++++++++++++++++++------------- drivers/video/fbdev/core/Makefile | 13 ++++--- include/linux/fb.h | 14 ++++++++ 6 files changed, 59 insertions(+), 31 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 44dd071d836..dc409539218 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -229,7 +229,7 @@ drivers-$(CONFIG_PCI) += arch/x86/pci/ # suspend and hibernation support drivers-$(CONFIG_PM) += arch/x86/power/ -drivers-$(CONFIG_FB) += arch/x86/video/ +drivers-$(CONFIG_FB_CORE) += arch/x86/video/ #### # boot loader support. Several targets are kept for legacy purposes diff --git a/arch/x86/video/Makefile b/arch/x86/video/Makefile index 11640c11611..cb0735bd662 100644 --- a/arch/x86/video/Makefile +++ b/arch/x86/video/Makefile @@ -1,2 +1,2 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_FB) += fbdev.o +obj-$(CONFIG_FB_CORE) += fbdev.o diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 840d9813b0b..0c562ed3495 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -71,7 +71,7 @@ config DUMMY_CONSOLE_ROWS config FRAMEBUFFER_CONSOLE bool "Framebuffer Console support" - depends on FB && !UML + depends on FB_CORE && !UML select VT_HW_CONSOLE_BINDING select CRC32 select FONT_SUPPORT diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig index d33c5cd684c..cb9e8b503a5 100644 --- a/drivers/video/fbdev/Kconfig +++ b/drivers/video/fbdev/Kconfig @@ -9,10 +9,8 @@ config FB_CMDLINE config FB_NOTIFY bool -menuconfig FB - tristate "Support for frame buffer devices" - select FB_CMDLINE - select FB_NOTIFY +menuconfig FB_CORE + tristate "Core support for frame buffer devices" help The frame buffer device provides an abstraction for the graphics hardware. It represents the frame buffer of some video hardware and @@ -36,6 +34,19 @@ menuconfig FB <http://www.munted.org.uk/programming/Framebuffer-HOWTO-1.3.html> for more information. + This options enables the core support for frame buffer devices. + +menuconfig FB + tristate "Support for frame buffer device drivers" + depends on FB_CORE + select FB_CMDLINE + select FB_NOTIFY + help + This enables support for frame buffer devices (fbdev) drivers. + + The DRM subsystem provides an option to emulate fbdev devices but + this option allows legacy fbdev drivers to be enabled as well. + Say Y here and to the driver for your graphics board below if you are compiling a kernel for a non-x86 architecture. @@ -47,7 +58,7 @@ menuconfig FB config FIRMWARE_EDID bool "Enable firmware EDID" - depends on FB + depends on FB_CORE help This enables access to the EDID transferred from the firmware. On the i386, this is from the Video BIOS. Enable this if DDC/I2C @@ -62,7 +73,7 @@ config FIRMWARE_EDID config FB_DDC tristate - depends on FB + depends on FB_CORE select I2C_ALGOBIT select I2C @@ -75,7 +86,7 @@ config FB_BOOT_VESA_SUPPORT config FB_CFB_FILLRECT tristate - depends on FB + depends on FB_CORE help Include the cfb_fillrect function for generic software rectangle filling. This is used by drivers that don't provide their own @@ -83,7 +94,7 @@ config FB_CFB_FILLRECT config FB_CFB_COPYAREA tristate - depends on FB + depends on FB_CORE help Include the cfb_copyarea function for generic software area copying. This is used by drivers that don't provide their own (accelerated) @@ -91,7 +102,7 @@ config FB_CFB_COPYAREA config FB_CFB_IMAGEBLIT tristate - depends on FB + depends on FB_CORE help Include the cfb_imageblit function for generic software image blitting. This is used by drivers that don't provide their own @@ -99,7 +110,7 @@ config FB_CFB_IMAGEBLIT config FB_CFB_REV_PIXELS_IN_BYTE bool - depends on FB + depends on FB_CORE help Allow generic frame-buffer functions to work on displays with 1, 2 and 4 bits per pixel depths which has opposite order of pixels in @@ -107,7 +118,7 @@ config FB_CFB_REV_PIXELS_IN_BYTE config FB_SYS_FILLRECT tristate - depends on FB + depends on FB_CORE help Include the sys_fillrect function for generic software rectangle filling. This is used by drivers that don't provide their own @@ -115,7 +126,7 @@ config FB_SYS_FILLRECT config FB_SYS_COPYAREA tristate - depends on FB + depends on FB_CORE help Include the sys_copyarea function for generic software area copying. This is used by drivers that don't provide their own (accelerated) @@ -123,7 +134,7 @@ config FB_SYS_COPYAREA config FB_SYS_IMAGEBLIT tristate - depends on FB + depends on FB_CORE help Include the sys_imageblit function for generic software image blitting. This is used by drivers that don't provide their own @@ -131,14 +142,14 @@ config FB_SYS_IMAGEBLIT config FB_PROVIDE_GET_FB_UNMAPPED_AREA bool - depends on FB + depends on FB_CORE help Allow generic frame-buffer to provide get_fb_unmapped_area function. menuconfig FB_FOREIGN_ENDIAN bool "Framebuffer foreign endianness support" - depends on FB + depends on FB_CORE help This menu will let you enable support for the framebuffers with non-native endianness (e.g. Little-Endian framebuffer on a @@ -162,36 +173,36 @@ endchoice config FB_SYS_FOPS tristate - depends on FB + depends on FB_CORE config FB_DEFERRED_IO bool - depends on FB + depends on FB_CORE config FB_HECUBA tristate - depends on FB + depends on FB_CORE depends on FB_DEFERRED_IO config FB_SVGALIB tristate - depends on FB + depends on FB_CORE help Common utility functions useful to fbdev drivers of VGA-based cards. config FB_MACMODES tristate - depends on FB + depends on FB_CORE config FB_BACKLIGHT tristate - depends on FB + depends on FB_CORE select BACKLIGHT_CLASS_DEVICE config FB_MODE_HELPERS bool "Enable Video Mode Handling Helpers" - depends on FB + depends on FB_CORE help This enables functions for handling video modes using the Generalized Timing Formula and the EDID parser. A few drivers rely @@ -201,7 +212,7 @@ config FB_MODE_HELPERS config FB_TILEBLITTING bool "Enable Tile Blitting Support" - depends on FB + depends on FB_CORE help This enables tile blitting. Tile blitting is a drawing technique where the screen is divided into rectangular sections (tiles), whereas diff --git a/drivers/video/fbdev/core/Makefile b/drivers/video/fbdev/core/Makefile index 26cbc965497..328a017c78b 100644 --- a/drivers/video/fbdev/core/Makefile +++ b/drivers/video/fbdev/core/Makefile @@ -1,22 +1,25 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_FB_CMDLINE) += fb_cmdline.o obj-$(CONFIG_FB_NOTIFY) += fb_notify.o +obj-$(CONFIG_FB_CORE) += fb_core.o obj-$(CONFIG_FB) += fb.o -fb-y := fbmem.o fbmon.o fbcmap.o fbsysfs.o \ +fb_core-y := fbmem.o fbmon.o fbcmap.o \ modedb.o fbcvt.o -fb-$(CONFIG_FB_DEFERRED_IO) += fb_defio.o +fb_core-$(CONFIG_FB_DEFERRED_IO) += fb_defio.o ifeq ($(CONFIG_FRAMEBUFFER_CONSOLE),y) -fb-y += fbcon.o bitblit.o softcursor.o +fb_core-y += fbcon.o bitblit.o softcursor.o ifeq ($(CONFIG_FB_TILEBLITTING),y) -fb-y += tileblit.o +fb_core-y += tileblit.o endif ifeq ($(CONFIG_FRAMEBUFFER_CONSOLE_ROTATION),y) -fb-y += fbcon_rotate.o fbcon_cw.o fbcon_ud.o \ +fb_core-y += fbcon_rotate.o fbcon_cw.o fbcon_ud.o \ fbcon_ccw.o endif endif +fb-y := fbsysfs.o + obj-$(CONFIG_FB_CFB_FILLRECT) += cfbfillrect.o obj-$(CONFIG_FB_CFB_COPYAREA) += cfbcopyarea.o obj-$(CONFIG_FB_CFB_IMAGEBLIT) += cfbimgblt.o diff --git a/include/linux/fb.h b/include/linux/fb.h index 96111248a25..f66eefbf1ee 100644 --- a/include/linux/fb.h +++ b/include/linux/fb.h @@ -689,9 +689,23 @@ static inline bool fb_be_math(struct fb_info *info) /* drivers/video/fbsysfs.c */ extern struct fb_info *framebuffer_alloc(size_t size, struct device *dev); extern void framebuffer_release(struct fb_info *info); +#if IS_ENABLED(CONFIG_FB) extern int fb_register_sysfs(struct fb_info *fb_info); extern void fb_unregister_sysfs(struct fb_info *head); extern void fb_bl_default_curve(struct fb_info *fb_info, u8 off, u8 min, u8 max); +#else +static inline int fb_register_sysfs(struct fb_info *fb_info) +{ + return 0; +} +static inline void fb_unregister_sysfs(struct fb_info *head) +{ +} +static inline void fb_bl_default_curve(struct fb_info *fb_info, u8 off, + u8 min, u8 max) +{ +} +#endif /* drivers/video/fbmon.c */ #define FB_MAXTIMINGS 0 -- 2.31.1

1 0

[RFC PATCH 2/4] fbdev: Move framebuffer_{alloc, release}() functions to fbmem.c
by Javier Martinez Canillas 27 Aug '21

27 Aug '21

The framebuffer_alloc() and framebuffer_release() functions are much more related with the functions in drivers/fbdev/core/fbmem.c than the ones in drivers/fbdev/core/fbsysfs.c, that are only to manage sysfs attributes. Signed-off-by: Javier Martinez Canillas <javierm(a)redhat.com> --- drivers/video/fbdev/core/fbmem.c | 65 +++++++++++++++++++++++++++ drivers/video/fbdev/core/fbsysfs.c | 71 ------------------------------ 2 files changed, 65 insertions(+), 71 deletions(-) diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c index d886582c0a4..2b22e46b815 100644 --- a/drivers/video/fbdev/core/fbmem.c +++ b/drivers/video/fbdev/core/fbmem.c @@ -57,6 +57,71 @@ bool fb_center_logo __read_mostly; int fb_logo_count __read_mostly = -1; +/** + * framebuffer_alloc - creates a new frame buffer info structure + * + * @size: size of driver private data, can be zero + * @dev: pointer to the device for this fb, this can be NULL + * + * Creates a new frame buffer info structure. Also reserves @size bytes + * for driver private data (info->par). info->par (if any) will be + * aligned to sizeof(long). + * + * Returns the new structure, or NULL if an error occurred. + * + */ +struct fb_info *framebuffer_alloc(size_t size, struct device *dev) +{ +#define BYTES_PER_LONG (BITS_PER_LONG/8) +#define PADDING (BYTES_PER_LONG - (sizeof(struct fb_info) % BYTES_PER_LONG)) + int fb_info_size = sizeof(struct fb_info); + struct fb_info *info; + char *p; + + if (size) + fb_info_size += PADDING; + + p = kzalloc(fb_info_size + size, GFP_KERNEL); + + if (!p) + return NULL; + + info = (struct fb_info *) p; + + if (size) + info->par = p + fb_info_size; + + info->device = dev; + info->fbcon_rotate_hint = -1; + +#if IS_ENABLED(CONFIG_FB_BACKLIGHT) + mutex_init(&info->bl_curve_mutex); +#endif + + return info; +#undef PADDING +#undef BYTES_PER_LONG +} +EXPORT_SYMBOL(framebuffer_alloc); + +/** + * framebuffer_release - marks the structure available for freeing + * + * @info: frame buffer info structure + * + * Drop the reference count of the device embedded in the + * framebuffer info structure. + * + */ +void framebuffer_release(struct fb_info *info) +{ + if (!info) + return; + kfree(info->apertures); + kfree(info); +} +EXPORT_SYMBOL(framebuffer_release); + static struct fb_info *get_fb_info(unsigned int idx) { struct fb_info *fb_info; diff --git a/drivers/video/fbdev/core/fbsysfs.c b/drivers/video/fbdev/core/fbsysfs.c index a040d6bd6c3..2c1e3f0effe 100644 --- a/drivers/video/fbdev/core/fbsysfs.c +++ b/drivers/video/fbdev/core/fbsysfs.c @@ -5,12 +5,6 @@ * Copyright (c) 2004 James Simmons <jsimmons(a)infradead.org> */ -/* - * Note: currently there's only stubs for framebuffer_alloc and - * framebuffer_release here. The reson for that is that until all drivers - * are converted to use it a sysfsification will open OOPSable races. - */ - #include <linux/kernel.h> #include <linux/slab.h> #include <linux/fb.h> @@ -20,71 +14,6 @@ #define FB_SYSFS_FLAG_ATTR 1 -/** - * framebuffer_alloc - creates a new frame buffer info structure - * - * @size: size of driver private data, can be zero - * @dev: pointer to the device for this fb, this can be NULL - * - * Creates a new frame buffer info structure. Also reserves @size bytes - * for driver private data (info->par). info->par (if any) will be - * aligned to sizeof(long). - * - * Returns the new structure, or NULL if an error occurred. - * - */ -struct fb_info *framebuffer_alloc(size_t size, struct device *dev) -{ -#define BYTES_PER_LONG (BITS_PER_LONG/8) -#define PADDING (BYTES_PER_LONG - (sizeof(struct fb_info) % BYTES_PER_LONG)) - int fb_info_size = sizeof(struct fb_info); - struct fb_info *info; - char *p; - - if (size) - fb_info_size += PADDING; - - p = kzalloc(fb_info_size + size, GFP_KERNEL); - - if (!p) - return NULL; - - info = (struct fb_info *) p; - - if (size) - info->par = p + fb_info_size; - - info->device = dev; - info->fbcon_rotate_hint = -1; - -#if IS_ENABLED(CONFIG_FB_BACKLIGHT) - mutex_init(&info->bl_curve_mutex); -#endif - - return info; -#undef PADDING -#undef BYTES_PER_LONG -} -EXPORT_SYMBOL(framebuffer_alloc); - -/** - * framebuffer_release - marks the structure available for freeing - * - * @info: frame buffer info structure - * - * Drop the reference count of the device embedded in the - * framebuffer info structure. - * - */ -void framebuffer_release(struct fb_info *info) -{ - if (!info) - return; - kfree(info->apertures); - kfree(info); -} -EXPORT_SYMBOL(framebuffer_release); - static int activate(struct fb_info *fb_info, struct fb_var_screeninfo *var) { int err; -- 2.31.1

1 0

[PATCH] drm/i915: Actually delete gpu reloc selftests
by Daniel Vetter 27 Aug '21

27 Aug '21

In commit 8e02cceb1f1f4f254625e5338dd997ff61ab40d7 Author: Daniel Vetter <daniel.vetter(a)ffwll.ch> Date: Tue Aug 3 14:48:33 2021 +0200 drm/i915: delete gpu reloc code I deleted the gpu relocation code and the selftest include and enabling, but accidentally forgot about the selftest source code. Fix this oversight. Signed-off-by: Daniel Vetter <daniel.vetter(a)intel.com> Cc: Jon Bloomfield <jon.bloomfield(a)intel.com> Cc: Chris Wilson <chris(a)chris-wilson.co.uk> Cc: Maarten Lankhorst <maarten.lankhorst(a)linux.intel.com> Cc: Daniel Vetter <daniel.vetter(a)ffwll.ch> Cc: Joonas Lahtinen <joonas.lahtinen(a)linux.intel.com> Cc: "Thomas Hellström" <thomas.hellstrom(a)linux.intel.com> Cc: Matthew Auld <matthew.auld(a)intel.com> Cc: Lionel Landwerlin <lionel.g.landwerlin(a)intel.com> Cc: Dave Airlie <airlied(a)redhat.com> Cc: Jason Ekstrand <jason(a)jlekstrand.net> --- .../i915/gem/selftests/i915_gem_execbuffer.c | 190 ------------------ 1 file changed, 190 deletions(-) delete mode 100644 drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c deleted file mode 100644 index 16162fc2782d..000000000000 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c +++ /dev/null @@ -1,190 +0,0 @@ -// SPDX-License-Identifier: MIT -/* - * Copyright © 2020 Intel Corporation - */ - -#include "i915_selftest.h" - -#include "gt/intel_engine_pm.h" -#include "selftests/igt_flush_test.h" - -static u64 read_reloc(const u32 *map, int x, const u64 mask) -{ - u64 reloc; - - memcpy(&reloc, &map[x], sizeof(reloc)); - return reloc & mask; -} - -static int __igt_gpu_reloc(struct i915_execbuffer *eb, - struct drm_i915_gem_object *obj) -{ - const unsigned int offsets[] = { 8, 3, 0 }; - const u64 mask = - GENMASK_ULL(eb->reloc_cache.use_64bit_reloc ? 63 : 31, 0); - const u32 *map = page_mask_bits(obj->mm.mapping); - struct i915_request *rq; - struct i915_vma *vma; - int err; - int i; - - vma = i915_vma_instance(obj, eb->context->vm, NULL); - if (IS_ERR(vma)) - return PTR_ERR(vma); - - err = i915_gem_object_lock(obj, &eb->ww); - if (err) - return err; - - err = i915_vma_pin_ww(vma, &eb->ww, 0, 0, PIN_USER | PIN_HIGH); - if (err) - return err; - - /* 8-Byte aligned */ - err = __reloc_entry_gpu(eb, vma, offsets[0] * sizeof(u32), 0); - if (err <= 0) - goto reloc_err; - - /* !8-Byte aligned */ - err = __reloc_entry_gpu(eb, vma, offsets[1] * sizeof(u32), 1); - if (err <= 0) - goto reloc_err; - - /* Skip to the end of the cmd page */ - i = PAGE_SIZE / sizeof(u32) - 1; - i -= eb->reloc_cache.rq_size; - memset32(eb->reloc_cache.rq_cmd + eb->reloc_cache.rq_size, - MI_NOOP, i); - eb->reloc_cache.rq_size += i; - - /* Force next batch */ - err = __reloc_entry_gpu(eb, vma, offsets[2] * sizeof(u32), 2); - if (err <= 0) - goto reloc_err; - - GEM_BUG_ON(!eb->reloc_cache.rq); - rq = i915_request_get(eb->reloc_cache.rq); - reloc_gpu_flush(eb, &eb->reloc_cache); - GEM_BUG_ON(eb->reloc_cache.rq); - - err = i915_gem_object_wait(obj, I915_WAIT_INTERRUPTIBLE, HZ / 2); - if (err) { - intel_gt_set_wedged(eb->engine->gt); - goto put_rq; - } - - if (!i915_request_completed(rq)) { - pr_err("%s: did not wait for relocations!\n", eb->engine->name); - err = -EINVAL; - goto put_rq; - } - - for (i = 0; i < ARRAY_SIZE(offsets); i++) { - u64 reloc = read_reloc(map, offsets[i], mask); - - if (reloc != i) { - pr_err("%s[%d]: map[%d] %llx != %x\n", - eb->engine->name, i, offsets[i], reloc, i); - err = -EINVAL; - } - } - if (err) - igt_hexdump(map, 4096); - -put_rq: - i915_request_put(rq); -unpin_vma: - i915_vma_unpin(vma); - return err; - -reloc_err: - if (!err) - err = -EIO; - goto unpin_vma; -} - -static int igt_gpu_reloc(void *arg) -{ - struct i915_execbuffer eb; - struct drm_i915_gem_object *scratch; - int err = 0; - u32 *map; - - eb.i915 = arg; - - scratch = i915_gem_object_create_internal(eb.i915, 4096); - if (IS_ERR(scratch)) - return PTR_ERR(scratch); - - map = i915_gem_object_pin_map_unlocked(scratch, I915_MAP_WC); - if (IS_ERR(map)) { - err = PTR_ERR(map); - goto err_scratch; - } - - intel_gt_pm_get(&eb.i915->gt); - - for_each_uabi_engine(eb.engine, eb.i915) { - if (intel_engine_requires_cmd_parser(eb.engine) || - intel_engine_using_cmd_parser(eb.engine)) - continue; - - reloc_cache_init(&eb.reloc_cache, eb.i915); - memset(map, POISON_INUSE, 4096); - - intel_engine_pm_get(eb.engine); - eb.context = intel_context_create(eb.engine); - if (IS_ERR(eb.context)) { - err = PTR_ERR(eb.context); - goto err_pm; - } - eb.reloc_pool = NULL; - eb.reloc_context = NULL; - - i915_gem_ww_ctx_init(&eb.ww, false); -retry: - err = intel_context_pin_ww(eb.context, &eb.ww); - if (!err) { - err = __igt_gpu_reloc(&eb, scratch); - - intel_context_unpin(eb.context); - } - if (err == -EDEADLK) { - err = i915_gem_ww_ctx_backoff(&eb.ww); - if (!err) - goto retry; - } - i915_gem_ww_ctx_fini(&eb.ww); - - if (eb.reloc_pool) - intel_gt_buffer_pool_put(eb.reloc_pool); - if (eb.reloc_context) - intel_context_put(eb.reloc_context); - - intel_context_put(eb.context); -err_pm: - intel_engine_pm_put(eb.engine); - if (err) - break; - } - - if (igt_flush_test(eb.i915)) - err = -EIO; - - intel_gt_pm_put(&eb.i915->gt); -err_scratch: - i915_gem_object_put(scratch); - return err; -} - -int i915_gem_execbuffer_live_selftests(struct drm_i915_private *i915) -{ - static const struct i915_subtest tests[] = { - SUBTEST(igt_gpu_reloc), - }; - - if (intel_gt_is_wedged(&i915->gt)) - return 0; - - return i915_live_subtests(tests, i915); -} -- 2.32.0

3 3

[PATCH v15 00/12] Restricted DMA
by Claire Chang 27 Aug '21

27 Aug '21

This series implements mitigations for lack of DMA access control on systems without an IOMMU, which could result in the DMA accessing the system memory at unexpected times and/or unexpected addresses, possibly leading to data leakage or corruption. For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is not behind an IOMMU. As PCI-e, by design, gives the device full access to system memory, a vulnerability in the Wi-Fi firmware could easily escalate to a full system exploit (remote wifi exploits: [1a], [1b] that shows a full chain of exploits; [2], [3]). To mitigate the security concerns, we introduce restricted DMA. Restricted DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a specially allocated region and does memory allocation from the same region. The feature on its own provides a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to restrict the DMA to a predefined memory region (this is usually done at firmware level, e.g. MPU in ATF on some ARM platforms [4]). [1a] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcom… [1b] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcom… [2] https://blade.tencent.com/en/advisories/qualpwn/ [3] https://www.bleepingcomputer.com/news/security/vulnerabilities-found-in-hig… [4] https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/media… v15: - Apply Will's diff (https://lore.kernel.org/patchwork/patch/1448957/#1647521) to fix the crash reported by Qian. - Add Stefano's Acked-by tag for patch 01/12 from v14 v14: - Move set_memory_decrypted before swiotlb_init_io_tlb_mem (patch 01/12, 10,12) - Add Stefano's Acked-by tag from v13 https://lore.kernel.org/patchwork/cover/1448954/ v13: - Fix xen-swiotlb issues - memset in patch 01/12 - is_swiotlb_force_bounce in patch 06/12 - Fix the dts example typo in reserved-memory.txt - Add Stefano and Will's Tested-by tag from v12 https://lore.kernel.org/patchwork/cover/1448001/ v12: Split is_dev_swiotlb_force into is_swiotlb_force_bounce (patch 06/12) and is_swiotlb_for_alloc (patch 09/12) https://lore.kernel.org/patchwork/cover/1447254/ v11: - Rebase against swiotlb devel/for-linus-5.14 - s/mempry/memory/g - exchange the order of patch 09/12 and 10/12 https://lore.kernel.org/patchwork/cover/1447216/ v10: Address the comments in v9 to - fix the dev->dma_io_tlb_mem assignment - propagate swiotlb_force setting into io_tlb_default_mem->force - move set_memory_decrypted out of swiotlb_init_io_tlb_mem - move debugfs_dir declaration into the main CONFIG_DEBUG_FS block - add swiotlb_ prefix to find_slots and release_slots - merge the 3 alloc/free related patches - move the CONFIG_DMA_RESTRICTED_POOL later https://lore.kernel.org/patchwork/cover/1446882/ v9: Address the comments in v7 to - set swiotlb active pool to dev->dma_io_tlb_mem - get rid of get_io_tlb_mem - dig out the device struct for is_swiotlb_active - move debugfs_create_dir out of swiotlb_create_debugfs - do set_memory_decrypted conditionally in swiotlb_init_io_tlb_mem - use IS_ENABLED in kernel/dma/direct.c - fix redefinition of 'of_dma_set_restricted_buffer' https://lore.kernel.org/patchwork/cover/1445081/ v8: - Fix reserved-memory.txt and add the reg property in example. - Fix sizeof for of_property_count_elems_of_size in drivers/of/address.c#of_dma_set_restricted_buffer. - Apply Will's suggestion to try the OF node having DMA configuration in drivers/of/address.c#of_dma_set_restricted_buffer. - Fix typo in the comment of drivers/of/address.c#of_dma_set_restricted_buffer. - Add error message for PageHighMem in kernel/dma/swiotlb.c#rmem_swiotlb_device_init and move it to rmem_swiotlb_setup. - Fix the message string in rmem_swiotlb_setup. https://lore.kernel.org/patchwork/cover/1437112/ v7: Fix debugfs, PageHighMem and comment style in rmem_swiotlb_device_init https://lore.kernel.org/patchwork/cover/1431031/ v6: Address the comments in v5 https://lore.kernel.org/patchwork/cover/1423201/ v5: Rebase on latest linux-next https://lore.kernel.org/patchwork/cover/1416899/ v4: - Fix spinlock bad magic - Use rmem->name for debugfs entry - Address the comments in v3 https://lore.kernel.org/patchwork/cover/1378113/ v3: Using only one reserved memory region for both streaming DMA and memory allocation. https://lore.kernel.org/patchwork/cover/1360992/ v2: Building on top of swiotlb. https://lore.kernel.org/patchwork/cover/1280705/ v1: Using dma_map_ops. https://lore.kernel.org/patchwork/cover/1271660/ Claire Chang (12): swiotlb: Refactor swiotlb init functions swiotlb: Refactor swiotlb_create_debugfs swiotlb: Set dev->dma_io_tlb_mem to the swiotlb pool used swiotlb: Update is_swiotlb_buffer to add a struct device argument swiotlb: Update is_swiotlb_active to add a struct device argument swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing swiotlb: Move alloc_size to swiotlb_find_slots swiotlb: Refactor swiotlb_tbl_unmap_single swiotlb: Add restricted DMA alloc/free support swiotlb: Add restricted DMA pool initialization dt-bindings: of: Add restricted DMA pool of: Add plumbing for restricted DMA pool .../reserved-memory/reserved-memory.txt | 36 ++- drivers/base/core.c | 4 + drivers/gpu/drm/i915/gem/i915_gem_internal.c | 2 +- drivers/gpu/drm/nouveau/nouveau_ttm.c | 2 +- drivers/iommu/dma-iommu.c | 12 +- drivers/of/address.c | 33 +++ drivers/of/device.c | 3 + drivers/of/of_private.h | 6 + drivers/pci/xen-pcifront.c | 2 +- drivers/xen/swiotlb-xen.c | 4 +- include/linux/device.h | 4 + include/linux/swiotlb.h | 53 +++- kernel/dma/Kconfig | 14 + kernel/dma/direct.c | 59 ++-- kernel/dma/direct.h | 8 +- kernel/dma/swiotlb.c | 251 +++++++++++++----- 16 files changed, 390 insertions(+), 103 deletions(-) -- 2.32.0.288.g62a8d224e6-goog

7 34

linux-next: manual merge of the vfio tree with the drm tree
by Stephen Rothwell 27 Aug '21

27 Aug '21

Hi all, Today's linux-next merge of the vfio tree got a conflict in: drivers/vfio/pci/vfio_pci.c between commits: b8779475869a ("vgaarb: provide a vga_client_unregister wrapper") f6b1772b2555 ("vgaarb: remove the unused irq_set_state argument to vga_client_register") bf44e8cecc03 ("vgaarb: don't pass a cookie to vga_client_register") from the drm tree and commits: 1cbd70fe3787 ("vfio/pci: Rename vfio_pci.c to vfio_pci_core.c") ff53edf6d6ab ("vfio/pci: Split the pci_driver code out of vfio_pci_core.c") from the vfio tree. I fixed it up (I reapplied the drm changes to vfio_pci_core.c - see below) and can carry the fix as necessary. This is now fixed as far as linux-next is concerned, but any non trivial conflicts should be mentioned to your upstream maintainer when your tree is submitted for merging. You may also want to consider cooperating with the maintainer of the conflicting tree to minimise any particularly complex conflicts. From: Stephen Rothwell <sfr(a)canb.auug.org.au> Date: Fri, 27 Aug 2021 16:52:45 +1000 Subject: [PATCH] fixup for "vfio/pci: Rename vfio_pci.c to vfio_pci_core.c" Signed-off-by: Stephen Rothwell <sfr(a)canb.auug.org.au> --- drivers/vfio/pci/vfio_pci_core.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 675616e08897..68198e0f2a63 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -53,10 +53,9 @@ static inline bool vfio_vga_disabled(void) * has no way to get to it and routing can be disabled externally at the * bridge. */ -static unsigned int vfio_pci_set_vga_decode(void *opaque, bool single_vga) +static unsigned int vfio_pci_set_decode(struct pci_dev *pdev, bool single_vga) { - struct vfio_pci_core_device *vdev = opaque; - struct pci_dev *tmp = NULL, *pdev = vdev->pdev; + struct pci_dev *tmp = NULL; unsigned char max_busnr; unsigned int decodes; @@ -1756,10 +1755,10 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) if (!vfio_pci_is_vga(pdev)) return 0; - ret = vga_client_register(pdev, vdev, NULL, vfio_pci_set_vga_decode); + ret = vga_client_register(pdev, vfio_pci_set_decode); if (ret) return ret; - vga_set_legacy_decoding(pdev, vfio_pci_set_vga_decode(vdev, false)); + vga_set_legacy_decoding(pdev, vfio_pci_set_decode(pdev, false)); return 0; } @@ -1769,7 +1768,7 @@ static void vfio_pci_vga_uninit(struct vfio_pci_core_device *vdev) if (!vfio_pci_is_vga(pdev)) return; - vga_client_register(pdev, NULL, NULL, NULL); + vga_client_unregister(pdev); vga_set_legacy_decoding(pdev, VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM | VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM); -- 2.32.0 -- Cheers, Stephen Rothwell

1 0

Adding sync_shrinkers function for TTM pool optimization
by Christian König 27 Aug '21

27 Aug '21

Hi Andrew, Daniel suggested that I ping you once more about this. Basically we want to add a barrier function to make sure that our TTM pool shrinker is not freeing up pages from a device while the device is being unplugged. Currently we are having a global mutex to serialize all of this, but this caused contention for unmapping the freed pages in the IOMMU. We just need your Acked-by and I hope my explanation is now more understandable than the last time. Cheers, Christian.

3 8

[PATCH V3 0/9] PCI/VGA: Rework default VGA device selection
by Huacai Chen 27 Aug '21

27 Aug '21

My original work is at [1]. Bjorn do some rework and extension in V2. It moves the VGA arbiter to the PCI subsystem, fixes a few nits, and breaks a few pieces to make the main patch a little smaller. V3 rewrite the commit log of the last patch (which is also summarized by Bjorn). All comments welcome! [1] https://lore.kernel.org/dri-devel/20210705100503.1120643-1-chenhuacai@loong… Bjorn Helgaas (4): PCI/VGA: Move vgaarb to drivers/pci PCI/VGA: Replace full MIT license text with SPDX identifier PCI/VGA: Use unsigned format string to print lock counts PCI/VGA: Remove empty vga_arb_device_card_gone() Huacai Chen (5): PCI/VGA: Move vga_arb_integrated_gpu() earlier in file PCI/VGA: Prefer vga_default_device() PCI/VGA: Split out vga_arb_update_default_device() PCI/VGA: Log bridge control messages when adding devices PCI/VGA: Rework default VGA device selection Signed-off-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Bjorn Helgaas <bhelgaas(a)google.com> --- drivers/gpu/vga/Kconfig | 19 --- drivers/gpu/vga/Makefile | 1 - drivers/pci/Kconfig | 19 +++ drivers/pci/Makefile | 1 + drivers/{gpu/vga => pci}/vgaarb.c | 269 ++++++++++++------------------ 5 files changed, 126 insertions(+), 183 deletions(-) rename drivers/{gpu/vga => pci}/vgaarb.c (90%) -- 2.27.0

3 11

[PATCH v2] drm/amd/pm: And destination bounds checking to struct copy
by Kees Cook 27 Aug '21

27 Aug '21

In preparation for FORTIFY_SOURCE performing compile-time and run-time field bounds checking for memcpy(), memmove(), and memset(), avoid intentionally writing across neighboring fields. The "Board Parameters" members of the structs: struct atom_smc_dpm_info_v4_5 struct atom_smc_dpm_info_v4_6 struct atom_smc_dpm_info_v4_7 struct atom_smc_dpm_info_v4_10 are written to the corresponding members of the corresponding PPTable_t variables, but they lack destination size bounds checking, which means the compiler cannot verify at compile time that this is an intended and safe memcpy(). Since the header files are effectively immutable[1] and a struct_group() cannot be used, nor a common struct referenced by both sides of the memcpy() arguments, add a new helper, amdgpu_memcpy_trailing(), to perform the bounds checking at compile time. Replace the open-coded memcpy()s with amdgpu_memcpy_trailing() which includes enough context for the bounds checking. "objdump -d" shows no object code changes. [1] https://lore.kernel.org/lkml/e56aad3c-a06f-da07-f491-a894a570d78f@amd.com Cc: "Christian König" <christian.koenig(a)amd.com> Cc: "Pan, Xinhui" <Xinhui.Pan(a)amd.com> Cc: David Airlie <airlied(a)linux.ie> Cc: Daniel Vetter <daniel(a)ffwll.ch> Cc: Hawking Zhang <Hawking.Zhang(a)amd.com> Cc: Feifei Xu <Feifei.Xu(a)amd.com> Cc: Likun Gao <Likun.Gao(a)amd.com> Cc: Jiawei Gu <Jiawei.Gu(a)amd.com> Cc: Evan Quan <evan.quan(a)amd.com> Cc: amd-gfx(a)lists.freedesktop.org Cc: dri-devel(a)lists.freedesktop.org Reviewed-by: Lijo Lazar <lijo.lazar(a)amd.com> Acked-by: Alex Deucher <alexander.deucher(a)amd.com> Signed-off-by: Kees Cook <keescook(a)chromium.org> --- v2: - rename and move helper to drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h - add reviews/acks v1: https://lore.kernel.org/lkml/20210819201441.3545027-1-keescook@chromium.org/ --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h | 24 +++++++++++++++++++ .../gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c | 6 ++--- .../gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c | 8 +++---- .../drm/amd/pm/swsmu/smu13/aldebaran_ppt.c | 5 ++-- 5 files changed, 33 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index dc3c6b3a00e5..c911387045e2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1452,4 +1452,5 @@ static inline int amdgpu_in_reset(struct amdgpu_device *adev) { return atomic_read(&adev->in_gpu_reset); } + #endif diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h index 715b4225f5ee..29031eb11d39 100644 --- a/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h +++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h @@ -1335,6 +1335,30 @@ enum smu_cmn2asic_mapping_type { #define WORKLOAD_MAP(profile, workload) \ [profile] = {1, (workload)} +/** + * amdgpu_memcpy_trailing - Copy the end of one structure into the middle of another + * + * @dst: Pointer to destination struct + * @first_dst_member: The member name in @dst where the overwrite begins + * @last_dst_member: The member name in @dst where the overwrite ends after + * @src: Pointer to the source struct + * @first_src_member: The member name in @src where the copy begins + * + */ +#define amdgpu_memcpy_trailing(dst, first_dst_member, last_dst_member, \ + src, first_src_member) \ +({ \ + size_t __src_offset = offsetof(typeof(*(src)), first_src_member); \ + size_t __src_size = sizeof(*(src)) - __src_offset; \ + size_t __dst_offset = offsetof(typeof(*(dst)), first_dst_member); \ + size_t __dst_size = offsetofend(typeof(*(dst)), last_dst_member) - \ + __dst_offset; \ + BUILD_BUG_ON(__src_size != __dst_size); \ + __builtin_memcpy((u8 *)(dst) + __dst_offset, \ + (u8 *)(src) + __src_offset, \ + __dst_size); \ +}) + #if !defined(SWSMU_CODE_LAYER_L2) && !defined(SWSMU_CODE_LAYER_L3) && !defined(SWSMU_CODE_LAYER_L4) int smu_get_power_limit(void *handle, uint32_t *limit, diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c index 273df66cac14..bda8fc12c91f 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c @@ -483,10 +483,8 @@ static int arcturus_append_powerplay_table(struct smu_context *smu) if ((smc_dpm_table->table_header.format_revision == 4) && (smc_dpm_table->table_header.content_revision == 6)) - memcpy(&smc_pptable->MaxVoltageStepGfx, - &smc_dpm_table->maxvoltagestepgfx, - sizeof(*smc_dpm_table) - offsetof(struct atom_smc_dpm_info_v4_6, maxvoltagestepgfx)); - + amdgpu_memcpy_trailing(smc_pptable, MaxVoltageStepGfx, BoardReserved, + smc_dpm_table, maxvoltagestepgfx); return 0; } diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c index f96681700c41..88a4a2aed48e 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c @@ -431,16 +431,16 @@ static int navi10_append_powerplay_table(struct smu_context *smu) switch (smc_dpm_table->table_header.content_revision) { case 5: /* nv10 and nv14 */ - memcpy(smc_pptable->I2cControllers, smc_dpm_table->I2cControllers, - sizeof(*smc_dpm_table) - sizeof(smc_dpm_table->table_header)); + amdgpu_memcpy_trailing(smc_pptable, I2cControllers, BoardReserved, + smc_dpm_table, I2cControllers); break; case 7: /* nv12 */ ret = amdgpu_atombios_get_data_table(adev, index, NULL, NULL, NULL, (uint8_t **)&smc_dpm_table_v4_7); if (ret) return ret; - memcpy(smc_pptable->I2cControllers, smc_dpm_table_v4_7->I2cControllers, - sizeof(*smc_dpm_table_v4_7) - sizeof(smc_dpm_table_v4_7->table_header)); + amdgpu_memcpy_trailing(smc_pptable, I2cControllers, BoardReserved, + smc_dpm_table_v4_7, I2cControllers); break; default: dev_err(smu->adev->dev, "smc_dpm_info with unsupported content revision %d!\n", diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c index ec8c30daf31c..d46b892846f6 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c @@ -409,9 +409,8 @@ static int aldebaran_append_powerplay_table(struct smu_context *smu) if ((smc_dpm_table->table_header.format_revision == 4) && (smc_dpm_table->table_header.content_revision == 10)) - memcpy(&smc_pptable->GfxMaxCurrent, - &smc_dpm_table->GfxMaxCurrent, - sizeof(*smc_dpm_table) - offsetof(struct atom_smc_dpm_info_v4_10, GfxMaxCurrent)); + amdgpu_memcpy_trailing(smc_pptable, GfxMaxCurrent, reserved, + smc_dpm_table, GfxMaxCurrent); return 0; } -- 2.30.2

2 2

← Newer
1
...
26
27
28
29
30
31
32
...
77
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

dri-devel August 2021