[RFC PATCH] drm/ttm: Fix swapping dereferences of freed memory

[Christian König]

Am 28.05.21 um 09:33 schrieb Thomas Hellström:
> On Fri, 2021-05-28 at 09:16 +0200, Christian König wrote:
>> Am 27.05.21 um 17:51 schrieb Thomas Hellström:
>>> On Thu, 2021-05-27 at 17:32 +0200, Christian König wrote:
>>>> Am 27.05.21 um 17:05 schrieb Thomas Hellström:
>>>>> On Thu, 2021-05-27 at 17:01 +0200, Thomas Hellström wrote:
>>>>>> On Thu, 2021-05-27 at 16:54 +0200, Christian König wrote:
>>>>>>> Am 27.05.21 um 16:19 schrieb Thomas Hellström:
>>>>>>>> The swapping code was dereference bo->ttm pointers
>>>>>>>> without
>>>>>>>> having
>>>>>>>> the
>>>>>>>> dma-resv lock held. Also it might try to swap out
>>>>>>>> unpopulated
>>>>>>>> bos.
>>>>>>>>
>>>>>>>> Fix this by moving the bo->ttm dereference until we have
>>>>>>>> the
>>>>>>>> reservation
>>>>>>>> lock. Check that the ttm_tt is populated after the
>>>>>>>> swap_notify
>>>>>>>> callback.
>>>>>>>>
>>>>>>>> Signed-off-by: Thomas Hellström
>>>>>>>> <thomas.hellstrom at linux.intel.com>
>>>>>>>> ---
>>>>>>>>      drivers/gpu/drm/ttm/ttm_bo.c     | 16
>>>>>>>> +++++++++++++++-
>>>>>>>>      drivers/gpu/drm/ttm/ttm_device.c |  8 +++-----
>>>>>>>>      2 files changed, 18 insertions(+), 6 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/gpu/drm/ttm/ttm_bo.c
>>>>>>>> b/drivers/gpu/drm/ttm/ttm_bo.c
>>>>>>>> index 9f53506a82fc..86213d37657b 100644
>>>>>>>> --- a/drivers/gpu/drm/ttm/ttm_bo.c
>>>>>>>> +++ b/drivers/gpu/drm/ttm/ttm_bo.c
>>>>>>>> @@ -1163,6 +1163,16 @@ int ttm_bo_swapout(struct
>>>>>>>> ttm_buffer_object
>>>>>>>> *bo, struct ttm_operation_ctx *ctx,
>>>>>>>>            if (!ttm_bo_evict_swapout_allowable(bo, ctx,
>>>>>>>> &place,
>>>>>>>> &locked, NULL))
>>>>>>>>                    return -EBUSY;
>>>>>>>>      
>>>>>>>> +       dma_resv_assert_held(bo->base.resv);
>>>>>>>> +
>>>>>>>> +       if (!bo->ttm ||
>>>>>>>> +           bo->ttm->page_flags & TTM_PAGE_FLAG_SG ||
>>>>>>>> +           bo->ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)
>>>>>>>> {
>>>>>>>> +               if (locked)
>>>>>>>> +                       dma_resv_unlock(bo->base.resv);
>>>>>>>> +               return -EBUSY;
>>>>>>>> +       }
>>>>>>>> +
>>>>>>>>            if (!ttm_bo_get_unless_zero(bo)) {
>>>>>>>>                    if (locked)
>>>>>>>>                            dma_resv_unlock(bo->base.resv);
>>>>>>>> @@ -1215,7 +1225,8 @@ int ttm_bo_swapout(struct
>>>>>>>> ttm_buffer_object
>>>>>>>> *bo, struct ttm_operation_ctx *ctx,
>>>>>>>>            if (bo->bdev->funcs->swap_notify)
>>>>>>>>                    bo->bdev->funcs->swap_notify(bo);
>>>>>>>>      
>>>>>>>> -       ret = ttm_tt_swapout(bo->bdev, bo->ttm,
>>>>>>>> gfp_flags);
>>>>>>>> +       if (ttm_tt_is_populated(bo->ttm))
>>>>>>>> +               ret = ttm_tt_swapout(bo->bdev, bo->ttm,
>>>>>>>> gfp_flags);
>>>>>>> Exactly that is what I won't recommend. We would try to
>>>>>>> swap
>>>>>>> out
>>>>>>> the
>>>>>>> same BO over and over again with that.
>>>>>> But we wouldn't since the BO is taken off the LRU and never
>>>>>> re-
>>>>>> added,
>>>>>>
>>>>>>
>>>>> In fact, we'd probably might want to take the !bo->ttm bos off
>>>>> the
>>>>> LRU
>>>>> as well..
>>>> No, we don't want to take any BOs of the LRU unless they are
>>>> pinned.
>>>>
>>>> Adding a TT object or populating it doesn't necessarily put the
>>>> BO
>>>> back
>>>> to the LRU.
>>> OK, but swapped bos are also taken off the LRU list so these
>>> unpopulated bos are just taking the same path. Only difference to
>>> swapped is that they don't get read back on re-populate, but
>>> typically
>>> cleared.
>>>
>>> But what would be the point of keeping swapped-out bos on the LRU
>>> list?, particularly when we're iterating under a spinlock?
>>> Shouldn't we try to re-add to LRU (if not already on an LRU) just
>>> before populating? There aren't really that many calls in core TTM.
>> I want to avoid removing BOs from the LRU as much as possible since
>> we
>> forgot on multiple places that we want to re-add them.
>>
>> Conceptual I think the swapped BOs should have a separate memory
>> domain,
>> this way we can ignore them cleanly when swapping things out.
> Yes, that would of course work as well. Keeping them on the system LRU
> is IMO highly undesirable.
>
>> Going to pick this patch up, modifying it a bit more and then pushing
>> it
>> to drm-misc-fixes for upstreaming.
> OK, I dropped the TTM fix for the purge-in-swap-notify from the i915
> series, hoping that the reworked variant of this patch lands first.

You will still need to add the second ttm_tt_populated() check since I 
dropped that for the back which I want to push to -fixes.

Regards,
Christian.

>
> Thanks,
> Thomas
>
>> Thanks,
>> Christian.
>>
>>> /Thomas
>>>
>>>
>>>
>>>
>>>
>>>> Christian.
>>>>
>>>>> /Thomas
>>>>>
>