[PATCH v3 1/2] drm/buddy: Implement tracking clear page feature

Arunpravin Paneer Selvam arunpravin.paneerselvam at amd.com
Thu Feb 8 15:47:47 UTC 2024



On 1/31/2024 11:59 PM, Matthew Auld wrote:
> On 30/01/2024 20:30, Arunpravin Paneer Selvam wrote:
>> Hi Matthew,
>>
>> On 12/21/2023 12:51 AM, Matthew Auld wrote:
>>> Hi,
>>>
>>> On 14/12/2023 13:42, Arunpravin Paneer Selvam wrote:
>>>> - Add tracking clear page feature.
>>>>
>>>> - Driver should enable the DRM_BUDDY_CLEARED flag if it
>>>>    successfully clears the blocks in the free path. On the otherhand,
>>>>    DRM buddy marks each block as cleared.
>>>>
>>>> - Track the available cleared pages size
>>>>
>>>> - If driver requests cleared memory we prefer cleared memory
>>>>    but fallback to uncleared if we can't find the cleared blocks.
>>>>    when driver requests uncleared memory we try to use uncleared but
>>>>    fallback to cleared memory if necessary.
>>>>
>>>> - When a block gets freed we clear it and mark the freed block as 
>>>> cleared,
>>>>    when there are buddies which are cleared as well we can merge them.
>>>>    Otherwise, we prefer to keep the blocks as separated.
>>>
>>> I was not involved, but it looks like we have also tried enabling 
>>> the clear-on-free idea for VRAM in i915 and then also tracking that 
>>> in the allocator, however that work unfortunately is not upstream. 
>>> The code is open source though: 
>>> https://github.com/intel-gpu/intel-gpu-i915-backports/blob/backport/main/drivers/gpu/drm/i915/i915_buddy.c#L300 
>>>
>>>
>>> It looks like some of the design differences there are having two 
>>> separate free lists, so mm->clean and mm->dirty (sounds reasonable 
>>> to me). And also the inclusion of a de-fragmentation routine, since 
>>> buddy blocks are now not always merged back, we might choose to run 
>>> the defrag in some cases, which also sounds reasonable. IIRC in 
>>> amdgpu userspace can control the page-size for an allocation, so 
>>> perhaps you would want to run it first if the allocation fails, 
>>> before trying to evict stuff?
>> I checked the clear-on-free idea implemented in i915. In amdgpu 
>> version, we are clearing all the blocks in amdgpu free routine and 
>> DRM buddy expects only the DRM_BUDDY_CLEARED flag. Basically, we are 
>> keeping the cleared blocks ready to be allocated when the user 
>> request for the cleared memory. We observed that this improves the 
>> performance on games and resolves the stutter issues as well. I see 
>> i915 active fences part does the same job for i915. Could we move 
>> this part into i915 free routine and set the DRM_BUDDY_CLEARED flag.
>>
>> On de-fragmentation , I have included a function which can be called 
>> at places where we get -ENOSPC. This routine will merge back the 
>> clear and dirty blocks together to form a larger block of requested 
>> size. I am wondering where we could use this routine as for the 
>> non-contiguous memory we have the fallback method and for the 
>> contiguous memory we have the try harder method which searches 
>> through the tree.
>
> Don't you also want to call it from your vram manager when the 
> requested page size is something large, before trying to evict stuff? 
> That could now fail due to fragmention IIUC. Or am I misreading 
> mdgpu_vram_mgr_new()?
Yes you are right, we can call the defragmentation routine from VRAM 
manager when there is a allocation failure.

Thanks,
Arun
>
>>
>> I agree we can have 2 lists (clear list and dirty list) and this 
>> would reduce the search iterations. But we need to handle the 2 lists 
>> design in all the functions which might require more time for testing 
>> on all platforms. Could we just go ahead with 1 list (free list) for 
>> now and I am going to take up this work as my next task.
>
> Sounds good.
>
>>
>> Thanks,
>> Arun.
>>>
>>>>
>>>> v1: (Christian)
>>>>    - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as
>>>>      cleared. Else, reset the clear flag for each block in the list.
>>>>
>>>>    - For merging the 2 cleared blocks compare as below,
>>>>      drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)
>>>>
>>>> Signed-off-by: Arunpravin Paneer Selvam 
>>>> <Arunpravin.PaneerSelvam at amd.com>
>>>> Suggested-by: Christian König <christian.koenig at amd.com>
>>>> ---
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c  |   6 +-
>>>>   drivers/gpu/drm/drm_buddy.c                   | 169 
>>>> +++++++++++++++---
>>>>   drivers/gpu/drm/i915/i915_ttm_buddy_manager.c |   6 +-
>>>>   drivers/gpu/drm/tests/drm_buddy_test.c        |  10 +-
>>>>   include/drm/drm_buddy.h                       |  18 +-
>>>>   5 files changed, 168 insertions(+), 41 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
>>>> index 08916538a615..d0e199cc8f17 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
>>>> @@ -556,7 +556,7 @@ static int amdgpu_vram_mgr_new(struct 
>>>> ttm_resource_manager *man,
>>>>       return 0;
>>>>     error_free_blocks:
>>>> -    drm_buddy_free_list(mm, &vres->blocks);
>>>> +    drm_buddy_free_list(mm, &vres->blocks, 0);
>>>>       mutex_unlock(&mgr->lock);
>>>>   error_fini:
>>>>       ttm_resource_fini(man, &vres->base);
>>>> @@ -589,7 +589,7 @@ static void amdgpu_vram_mgr_del(struct 
>>>> ttm_resource_manager *man,
>>>>         amdgpu_vram_mgr_do_reserve(man);
>>>>   -    drm_buddy_free_list(mm, &vres->blocks);
>>>> +    drm_buddy_free_list(mm, &vres->blocks, 0);
>>>>       mutex_unlock(&mgr->lock);
>>>>         atomic64_sub(vis_usage, &mgr->vis_usage);
>>>> @@ -897,7 +897,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device 
>>>> *adev)
>>>>           kfree(rsv);
>>>>         list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, 
>>>> blocks) {
>>>> -        drm_buddy_free_list(&mgr->mm, &rsv->allocated);
>>>> +        drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0);
>>>>           kfree(rsv);
>>>>       }
>>>>       if (!adev->gmc.is_app_apu)
>>>> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
>>>> index f57e6d74fb0e..d44172f23f05 100644
>>>> --- a/drivers/gpu/drm/drm_buddy.c
>>>> +++ b/drivers/gpu/drm/drm_buddy.c
>>>> @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy 
>>>> *mm,
>>>>       __list_add(&block->link, node->link.prev, &node->link);
>>>>   }
>>>>   +static void clear_reset(struct drm_buddy_block *block)
>>>> +{
>>>> +    block->header &= ~DRM_BUDDY_HEADER_CLEAR;
>>>> +}
>>>> +
>>>> +static void mark_cleared(struct drm_buddy_block *block)
>>>> +{
>>>> +    block->header |= DRM_BUDDY_HEADER_CLEAR;
>>>> +}
>>>> +
>>>>   static void mark_allocated(struct drm_buddy_block *block)
>>>>   {
>>>>       block->header &= ~DRM_BUDDY_HEADER_STATE;
>>>> @@ -223,6 +233,12 @@ static int split_block(struct drm_buddy *mm,
>>>>       mark_free(mm, block->left);
>>>>       mark_free(mm, block->right);
>>>>   +    if (drm_buddy_block_is_clear(block)) {
>>>> +        mark_cleared(block->left);
>>>> +        mark_cleared(block->right);
>>>> +        clear_reset(block);
>>>> +    }
>>>> +
>>>>       mark_split(block);
>>>>         return 0;
>>>> @@ -273,6 +289,13 @@ static void __drm_buddy_free(struct drm_buddy 
>>>> *mm,
>>>>           if (!drm_buddy_block_is_free(buddy))
>>>>               break;
>>>>   +        if (drm_buddy_block_is_clear(block) !=
>>>> +            drm_buddy_block_is_clear(buddy))
>>>> +            break;
>>>> +
>>>> +        if (drm_buddy_block_is_clear(block))
>>>> +            mark_cleared(parent);
>>>> +
>>>>           list_del(&buddy->link);
>>>>             drm_block_free(mm, block);
>>>> @@ -295,6 +318,9 @@ void drm_buddy_free_block(struct drm_buddy *mm,
>>>>   {
>>>>       BUG_ON(!drm_buddy_block_is_allocated(block));
>>>>       mm->avail += drm_buddy_block_size(mm, block);
>>>> +    if (drm_buddy_block_is_clear(block))
>>>> +        mm->clear_avail += drm_buddy_block_size(mm, block);
>>>> +
>>>>       __drm_buddy_free(mm, block);
>>>>   }
>>>>   EXPORT_SYMBOL(drm_buddy_free_block);
>>>> @@ -305,10 +331,20 @@ EXPORT_SYMBOL(drm_buddy_free_block);
>>>>    * @mm: DRM buddy manager
>>>>    * @objects: input list head to free blocks
>>>>    */
>>>> -void drm_buddy_free_list(struct drm_buddy *mm, struct list_head 
>>>> *objects)
>>>> +void drm_buddy_free_list(struct drm_buddy *mm,
>>>> +             struct list_head *objects,
>>>> +             unsigned long flags)
>>>>   {
>>>>       struct drm_buddy_block *block, *on;
>>>>   +    if (flags & DRM_BUDDY_CLEARED) {
>>>> +        list_for_each_entry(block, objects, link)
>>>> +            mark_cleared(block);
>>>> +    } else {
>>>> +        list_for_each_entry(block, objects, link)
>>>> +            clear_reset(block);
>>>> +    }
>>>> +
>>>>       list_for_each_entry_safe(block, on, objects, link) {
>>>>           drm_buddy_free_block(mm, block);
>>>>           cond_resched();
>>>> @@ -328,9 +364,11 @@ static inline bool contains(u64 s1, u64 e1, 
>>>> u64 s2, u64 e2)
>>>>   }
>>>>     static struct drm_buddy_block *
>>>> -alloc_range_bias(struct drm_buddy *mm,
>>>> -         u64 start, u64 end,
>>>> -         unsigned int order)
>>>> +__alloc_range_bias(struct drm_buddy *mm,
>>>> +           u64 start, u64 end,
>>>> +           unsigned int order,
>>>> +           unsigned long flags,
>>>> +           bool fallback)
>>>>   {
>>>>       struct drm_buddy_block *block;
>>>>       struct drm_buddy_block *buddy;
>>>> @@ -369,6 +407,15 @@ alloc_range_bias(struct drm_buddy *mm,
>>>>             if (contains(start, end, block_start, block_end) &&
>>>>               order == drm_buddy_block_order(block)) {
>>>> +            if (!fallback) {
>>>> +                if (flags & DRM_BUDDY_CLEAR_ALLOCATION) {
>>>> +                    if (!drm_buddy_block_is_clear(block))
>>>> +                        continue;
>>>> +                } else {
>>>> +                    if (drm_buddy_block_is_clear(block))
>>>> +                        continue;
>>>> +                }
>>>> +            }
>>>>               /*
>>>>                * Find the free block within the range.
>>>>                */
>>>> @@ -405,25 +452,58 @@ alloc_range_bias(struct drm_buddy *mm,
>>>>   }
>>>>     static struct drm_buddy_block *
>>>> -get_maxblock(struct drm_buddy *mm, unsigned int order)
>>>> +__drm_buddy_alloc_range_bias(struct drm_buddy *mm,
>>>> +                 u64 start, u64 end,
>>>> +                 unsigned int order,
>>>> +                 unsigned long flags)
>>>> +{
>>>> +    struct drm_buddy_block *block;
>>>> +    bool fallback = 0;
>>>> +
>>>> +    block = __alloc_range_bias(mm, start, end, order,
>>>> +                   flags, fallback);
>>>> +    if (IS_ERR(block))
>>>> +        return __alloc_range_bias(mm, start, end, order,
>>>> +                      flags, !fallback);
>>>> +
>>>> +    return block;
>>>> +}
>>>> +
>>>> +static struct drm_buddy_block *
>>>> +get_maxblock(struct drm_buddy *mm, unsigned int order,
>>>> +         unsigned long flags)
>>>>   {
>>>> -    struct drm_buddy_block *max_block = NULL, *node;
>>>> +    struct drm_buddy_block *max_block = NULL, *block = NULL;
>>>>       unsigned int i;
>>>>         for (i = order; i <= mm->max_order; ++i) {
>>>> -        if (!list_empty(&mm->free_list[i])) {
>>>> -            node = list_last_entry(&mm->free_list[i],
>>>> -                           struct drm_buddy_block,
>>>> -                           link);
>>>> -            if (!max_block) {
>>>> -                max_block = node;
>>>> -                continue;
>>>> +        struct drm_buddy_block *tmp_block;
>>>> +
>>>> +        list_for_each_entry_reverse(tmp_block, &mm->free_list[i], 
>>>> link) {
>>>> +            if (flags & DRM_BUDDY_CLEAR_ALLOCATION) {
>>>> +                /* Find a cleared block */
>>>> +                if (!drm_buddy_block_is_clear(tmp_block))
>>>> +                    continue;
>>>> +            } else {
>>>> +                if (drm_buddy_block_is_clear(tmp_block))
>>>> +                    continue;
>>>>               }
>>>>   -            if (drm_buddy_block_offset(node) >
>>>> -                drm_buddy_block_offset(max_block)) {
>>>> -                max_block = node;
>>>> -            }
>>>> +            block = tmp_block;
>>>> +            break;
>>>> +        }
>>>> +
>>>> +        if (!block)
>>>> +            continue;
>>>> +
>>>> +        if (!max_block) {
>>>> +            max_block = block;
>>>> +            continue;
>>>> +        }
>>>> +
>>>> +        if (drm_buddy_block_offset(block) >
>>>> +            drm_buddy_block_offset(max_block)) {
>>>> +            max_block = block;
>>>>           }
>>>>       }
>>>>   @@ -440,11 +520,35 @@ alloc_from_freelist(struct drm_buddy *mm,
>>>>       int err;
>>>>         if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) {
>>>> -        block = get_maxblock(mm, order);
>>>> +        block = get_maxblock(mm, order, flags);
>>>>           if (block)
>>>>               /* Store the obtained block order */
>>>>               tmp = drm_buddy_block_order(block);
>>>>       } else {
>>>> +        for (tmp = order; tmp <= mm->max_order; ++tmp) {
>>>> +            struct drm_buddy_block *tmp_block;
>>>> +
>>>> +            list_for_each_entry_reverse(tmp_block, 
>>>> &mm->free_list[tmp], link) {
>>>> +                if (flags & DRM_BUDDY_CLEAR_ALLOCATION) {
>>>> +                    /* Find a cleared block */
>>>> +                    if (!drm_buddy_block_is_clear(tmp_block))
>>>> +                        continue;
>>>> +                } else {
>>>> +                    if (drm_buddy_block_is_clear(tmp_block))
>>>> +                        continue;
>>>> +                }
>>>> +
>>>> +                block = tmp_block;
>>>> +                break;
>>>> +            }
>>>> +
>>>> +            if (block)
>>>> +                break;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    if (!block) {
>>>> +        /* Fallback method */
>>>>           for (tmp = order; tmp <= mm->max_order; ++tmp) {
>>>>               if (!list_empty(&mm->free_list[tmp])) {
>>>>                   block = list_last_entry(&mm->free_list[tmp],
>>>> @@ -454,10 +558,10 @@ alloc_from_freelist(struct drm_buddy *mm,
>>>>                       break;
>>>>               }
>>>>           }
>>>> -    }
>>>>   -    if (!block)
>>>> -        return ERR_PTR(-ENOSPC);
>>>> +        if (!block)
>>>> +            return ERR_PTR(-ENOSPC);
>>>> +    }
>>>>         BUG_ON(!drm_buddy_block_is_free(block));
>>>>   @@ -524,6 +628,8 @@ static int __alloc_range(struct drm_buddy *mm,
>>>>               mark_allocated(block);
>>>>               total_allocated += drm_buddy_block_size(mm, block);
>>>>               mm->avail -= drm_buddy_block_size(mm, block);
>>>> +            if (drm_buddy_block_is_clear(block))
>>>> +                mm->clear_avail -= drm_buddy_block_size(mm, block);
>>>>               list_add_tail(&block->link, &allocated);
>>>>               continue;
>>>>           }
>>>> @@ -558,7 +664,7 @@ static int __alloc_range(struct drm_buddy *mm,
>>>>           list_splice_tail(&allocated, blocks);
>>>>           *total_allocated_on_err = total_allocated;
>>>>       } else {
>>>> -        drm_buddy_free_list(mm, &allocated);
>>>> +        drm_buddy_free_list(mm, &allocated, 0);
>>>>       }
>>>>         return err;
>>>> @@ -624,11 +730,11 @@ static int __alloc_contig_try_harder(struct 
>>>> drm_buddy *mm,
>>>>               list_splice(&blocks_lhs, blocks);
>>>>               return 0;
>>>>           } else if (err != -ENOSPC) {
>>>> -            drm_buddy_free_list(mm, blocks);
>>>> +            drm_buddy_free_list(mm, blocks, 0);
>>>>               return err;
>>>>           }
>>>>           /* Free blocks for the next iteration */
>>>> -        drm_buddy_free_list(mm, blocks);
>>>> +        drm_buddy_free_list(mm, blocks, 0);
>>>>       }
>>>>         return -ENOSPC;
>>>> @@ -684,6 +790,8 @@ int drm_buddy_block_trim(struct drm_buddy *mm,
>>>>       list_del(&block->link);
>>>>       mark_free(mm, block);
>>>>       mm->avail += drm_buddy_block_size(mm, block);
>>>> +    if (drm_buddy_block_is_clear(block))
>>>> +        mm->clear_avail += drm_buddy_block_size(mm, block);
>>>>         /* Prevent recursively freeing this node */
>>>>       parent = block->parent;
>>>> @@ -695,6 +803,8 @@ int drm_buddy_block_trim(struct drm_buddy *mm,
>>>>       if (err) {
>>>>           mark_allocated(block);
>>>>           mm->avail -= drm_buddy_block_size(mm, block);
>>>> +        if (drm_buddy_block_is_clear(block))
>>>> +            mm->clear_avail -= drm_buddy_block_size(mm, block);
>>>>           list_add(&block->link, blocks);
>>>>       }
>>>>   @@ -782,7 +892,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
>>>>           do {
>>>>               if (flags & DRM_BUDDY_RANGE_ALLOCATION)
>>>>                   /* Allocate traversing within the range */
>>>> -                block = alloc_range_bias(mm, start, end, order);
>>>> +                block = __drm_buddy_alloc_range_bias(mm, start, end,
>>>> +                                     order, flags);
>>>>               else
>>>>                   /* Allocate from freelist */
>>>>                   block = alloc_from_freelist(mm, order, flags);
>>>> @@ -808,6 +919,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
>>>>             mark_allocated(block);
>>>>           mm->avail -= drm_buddy_block_size(mm, block);
>>>> +        if (drm_buddy_block_is_clear(block))
>>>> +            mm->clear_avail -= drm_buddy_block_size(mm, block);
>>>>           kmemleak_update_trace(block);
>>>>           list_add_tail(&block->link, &allocated);
>>>>   @@ -846,7 +959,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
>>>>       return 0;
>>>>     err_free:
>>>> -    drm_buddy_free_list(mm, &allocated);
>>>> +    drm_buddy_free_list(mm, &allocated, 0);
>>>>       return err;
>>>>   }
>>>>   EXPORT_SYMBOL(drm_buddy_alloc_blocks);
>>>> @@ -879,8 +992,8 @@ void drm_buddy_print(struct drm_buddy *mm, 
>>>> struct drm_printer *p)
>>>>   {
>>>>       int order;
>>>>   -    drm_printf(p, "chunk_size: %lluKiB, total: %lluMiB, free: 
>>>> %lluMiB\n",
>>>> -           mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20);
>>>> +    drm_printf(p, "chunk_size: %lluKiB, total: %lluMiB, free: 
>>>> %lluMiB, clear_free: %lluMiB\n",
>>>> +           mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, 
>>>> mm->clear_avail >> 20);
>>>>         for (order = mm->max_order; order >= 0; order--) {
>>>>           struct drm_buddy_block *block;
>>>> diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c 
>>>> b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
>>>> index 0d735d5c2b35..942345548bc3 100644
>>>> --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
>>>> +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
>>>> @@ -126,7 +126,7 @@ static int i915_ttm_buddy_man_alloc(struct 
>>>> ttm_resource_manager *man,
>>>>       return 0;
>>>>     err_free_blocks:
>>>> -    drm_buddy_free_list(mm, &bman_res->blocks);
>>>> +    drm_buddy_free_list(mm, &bman_res->blocks, 0);
>>>>       mutex_unlock(&bman->lock);
>>>>   err_free_res:
>>>>       ttm_resource_fini(man, &bman_res->base);
>>>> @@ -141,7 +141,7 @@ static void i915_ttm_buddy_man_free(struct 
>>>> ttm_resource_manager *man,
>>>>       struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
>>>>         mutex_lock(&bman->lock);
>>>> -    drm_buddy_free_list(&bman->mm, &bman_res->blocks);
>>>> +    drm_buddy_free_list(&bman->mm, &bman_res->blocks, 0);
>>>>       bman->visible_avail += bman_res->used_visible_size;
>>>>       mutex_unlock(&bman->lock);
>>>>   @@ -345,7 +345,7 @@ int i915_ttm_buddy_man_fini(struct ttm_device 
>>>> *bdev, unsigned int type)
>>>>       ttm_set_driver_manager(bdev, type, NULL);
>>>>         mutex_lock(&bman->lock);
>>>> -    drm_buddy_free_list(mm, &bman->reserved);
>>>> +    drm_buddy_free_list(mm, &bman->reserved, 0);
>>>>       drm_buddy_fini(mm);
>>>>       bman->visible_avail += bman->visible_reserved;
>>>>       WARN_ON_ONCE(bman->visible_avail != bman->visible_size);
>>>> diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c 
>>>> b/drivers/gpu/drm/tests/drm_buddy_test.c
>>>> index ea2af6bd9abe..e0860fce9ebd 100644
>>>> --- a/drivers/gpu/drm/tests/drm_buddy_test.c
>>>> +++ b/drivers/gpu/drm/tests/drm_buddy_test.c
>>>> @@ -83,7 +83,7 @@ static void 
>>>> drm_test_buddy_alloc_pathological(struct kunit *test)
>>>>                                 top, max_order);
>>>>       }
>>>>   -    drm_buddy_free_list(&mm, &holes);
>>>> +    drm_buddy_free_list(&mm, &holes, 0);
>>>>         /* Nothing larger than blocks of chunk_size now available */
>>>>       for (order = 1; order <= max_order; order++) {
>>>> @@ -95,7 +95,7 @@ static void 
>>>> drm_test_buddy_alloc_pathological(struct kunit *test)
>>>>       }
>>>>         list_splice_tail(&holes, &blocks);
>>>> -    drm_buddy_free_list(&mm, &blocks);
>>>> +    drm_buddy_free_list(&mm, &blocks, 0);
>>>>       drm_buddy_fini(&mm);
>>>>   }
>>>>   @@ -190,7 +190,7 @@ static void 
>>>> drm_test_buddy_alloc_pessimistic(struct kunit *test)
>>>>         list_del(&block->link);
>>>>       drm_buddy_free_block(&mm, block);
>>>> -    drm_buddy_free_list(&mm, &blocks);
>>>> +    drm_buddy_free_list(&mm, &blocks, 0);
>>>>       drm_buddy_fini(&mm);
>>>>   }
>>>>   @@ -236,7 +236,7 @@ static void 
>>>> drm_test_buddy_alloc_optimistic(struct kunit *test)
>>>>                                  size, size, &tmp, flags),
>>>>                             "buddy_alloc unexpectedly succeeded, it 
>>>> should be full!");
>>>>   -    drm_buddy_free_list(&mm, &blocks);
>>>> +    drm_buddy_free_list(&mm, &blocks, 0);
>>>>       drm_buddy_fini(&mm);
>>>>   }
>>>>   @@ -271,7 +271,7 @@ static void drm_test_buddy_alloc_limit(struct 
>>>> kunit *test)
>>>>                           drm_buddy_block_size(&mm, block),
>>>>                           BIT_ULL(mm.max_order) * PAGE_SIZE);
>>>>   -    drm_buddy_free_list(&mm, &allocated);
>>>> +    drm_buddy_free_list(&mm, &allocated, 0);
>>>>       drm_buddy_fini(&mm);
>>>>   }
>>>>   diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
>>>> index a5b39fc01003..f7311b59f2b0 100644
>>>> --- a/include/drm/drm_buddy.h
>>>> +++ b/include/drm/drm_buddy.h
>>>> @@ -6,6 +6,7 @@
>>>>   #ifndef __DRM_BUDDY_H__
>>>>   #define __DRM_BUDDY_H__
>>>>   +#include <linux/bitfield.h>
>>>>   #include <linux/bitops.h>
>>>>   #include <linux/list.h>
>>>>   #include <linux/slab.h>
>>>> @@ -25,15 +26,19 @@
>>>>   #define DRM_BUDDY_RANGE_ALLOCATION        BIT(0)
>>>>   #define DRM_BUDDY_TOPDOWN_ALLOCATION        BIT(1)
>>>>   #define DRM_BUDDY_CONTIGUOUS_ALLOCATION        BIT(2)
>>>> +#define DRM_BUDDY_CLEAR_ALLOCATION        BIT(3)
>>>> +#define DRM_BUDDY_CLEARED            BIT(4)
>>>>     struct drm_buddy_block {
>>>>   #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
>>>>   #define DRM_BUDDY_HEADER_STATE  GENMASK_ULL(11, 10)
>>>> +#define DRM_BUDDY_HEADER_CLEAR  GENMASK_ULL(9, 9)
>>>> +
>>>>   #define   DRM_BUDDY_ALLOCATED       (1 << 10)
>>>>   #define   DRM_BUDDY_FREE       (2 << 10)
>>>>   #define   DRM_BUDDY_SPLIT       (3 << 10)
>>>>   /* Free to be used, if needed in the future */
>>>> -#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(9, 6)
>>>> +#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6)
>>>>   #define DRM_BUDDY_HEADER_ORDER  GENMASK_ULL(5, 0)
>>>>       u64 header;
>>>>   @@ -86,6 +91,7 @@ struct drm_buddy {
>>>>       u64 chunk_size;
>>>>       u64 size;
>>>>       u64 avail;
>>>> +    u64 clear_avail;
>>>>   };
>>>>     static inline u64
>>>> @@ -112,6 +118,12 @@ drm_buddy_block_is_allocated(struct 
>>>> drm_buddy_block *block)
>>>>       return drm_buddy_block_state(block) == DRM_BUDDY_ALLOCATED;
>>>>   }
>>>>   +static inline bool
>>>> +drm_buddy_block_is_clear(struct drm_buddy_block *block)
>>>> +{
>>>> +    return block->header & DRM_BUDDY_HEADER_CLEAR;
>>>> +}
>>>> +
>>>>   static inline bool
>>>>   drm_buddy_block_is_free(struct drm_buddy_block *block)
>>>>   {
>>>> @@ -150,7 +162,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm,
>>>>     void drm_buddy_free_block(struct drm_buddy *mm, struct 
>>>> drm_buddy_block *block);
>>>>   -void drm_buddy_free_list(struct drm_buddy *mm, struct list_head 
>>>> *objects);
>>>> +void drm_buddy_free_list(struct drm_buddy *mm,
>>>> +             struct list_head *objects,
>>>> +             unsigned long flags);
>>>>     void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p);
>>>>   void drm_buddy_block_print(struct drm_buddy *mm,
>>



More information about the amd-gfx mailing list