[PATCH] gpu: drm: remove redundant dma_fence_put() when drm_sched_job_add_dependency() fails
Andrey Grodzovsky
andrey.grodzovsky at amd.com
Thu Apr 28 15:27:08 UTC 2022
On 2022-04-28 04:56, Hangyu Hua wrote:
> On 2022/4/27 22:43, Andrey Grodzovsky wrote:
>>
>> On 2022-04-26 22:31, Hangyu Hua wrote:
>>> On 2022/4/26 22:55, Andrey Grodzovsky wrote:
>>>>
>>>> On 2022-04-25 22:54, Hangyu Hua wrote:
>>>>> On 2022/4/25 23:42, Andrey Grodzovsky wrote:
>>>>>> On 2022-04-25 04:36, Hangyu Hua wrote:
>>>>>>
>>>>>>> When drm_sched_job_add_dependency() fails, dma_fence_put() will
>>>>>>> be called
>>>>>>> internally. Calling it again after
>>>>>>> drm_sched_job_add_dependency() finishes
>>>>>>> may result in a dangling pointer.
>>>>>>>
>>>>>>> Fix this by removing redundant dma_fence_put().
>>>>>>>
>>>>>>> Signed-off-by: Hangyu Hua <hbh25y at gmail.com>
>>>>>>> ---
>>>>>>> drivers/gpu/drm/lima/lima_gem.c | 1 -
>>>>>>> drivers/gpu/drm/scheduler/sched_main.c | 1 -
>>>>>>> 2 files changed, 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/lima/lima_gem.c
>>>>>>> b/drivers/gpu/drm/lima/lima_gem.c
>>>>>>> index 55bb1ec3c4f7..99c8e7f6bb1c 100644
>>>>>>> --- a/drivers/gpu/drm/lima/lima_gem.c
>>>>>>> +++ b/drivers/gpu/drm/lima/lima_gem.c
>>>>>>> @@ -291,7 +291,6 @@ static int lima_gem_add_deps(struct drm_file
>>>>>>> *file, struct lima_submit *submit)
>>>>>>> err =
>>>>>>> drm_sched_job_add_dependency(&submit->task->base, fence);
>>>>>>> if (err) {
>>>>>>> - dma_fence_put(fence);
>>>>>>> return err;
>>>>>>
>>>>>>
>>>>>> Makes sense here
>>>>>>
>>>>>>
>>>>>>> }
>>>>>>> }
>>>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c
>>>>>>> b/drivers/gpu/drm/scheduler/sched_main.c
>>>>>>> index b81fceb0b8a2..ebab9eca37a8 100644
>>>>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>>>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>>>>>> @@ -708,7 +708,6 @@ int
>>>>>>> drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
>>>>>>> dma_fence_get(fence);
>>>>>>> ret = drm_sched_job_add_dependency(job, fence);
>>>>>>> if (ret) {
>>>>>>> - dma_fence_put(fence);
>>>>>>
>>>>>>
>>>>>>
>>>>>> Not sure about this one since if you look at the relevant commits -
>>>>>> 'drm/scheduler: fix drm_sched_job_add_implicit_dependencies' and
>>>>>> 'drm/scheduler: fix drm_sched_job_add_implicit_dependencies harder'
>>>>>> You will see that the dma_fence_put here balances the extra
>>>>>> dma_fence_get
>>>>>> above
>>>>>>
>>>>>> Andrey
>>>>>>
>>>>>
>>>>> I don't think so. I checked the call chain and found no additional
>>>>> dma_fence_get(). But dma_fence_get() needs to be called before
>>>>> drm_sched_job_add_dependency() to keep the counter balanced.
>>>>
>>>>
>>>> I don't say there is an additional get, I just say that
>>>> drm_sched_job_add_dependency doesn't grab an extra reference to the
>>>> fences it stores so this needs to be done outside and for that
>>>> drm_sched_job_add_implicit_dependencies->dma_fence_get is called
>>>> and, if this addition fails you just call dma_fence_put to keep the
>>>> counter balanced.
>>>>
>>>
>>> drm_sched_job_add_implicit_dependencies() will call
>>> drm_sched_job_add_dependency(). And drm_sched_job_add_dependency()
>>> already call dma_fence_put() when it fails. Calling dma_fence_put()
>>> twice doesn't make sense.
>>>
>>> dma_fence_get() is in [2]. But dma_fence_put() will be called in [1]
>>> and [3] when xa_alloc() fails.
>>
>>
>> The way I see it, [2] and [3] are mat matching *get* and *put*
>> respectively. [1] *put* is against the original
>> dma_fence_init->kref_init of the fence which always set the refcount
>> at 1.
>> Also in support of this see commit 'drm/scheduler: fix
>> drm_sched_job_add_implicit_dependencies harder' - it says there
>> "drm_sched_job_add_dependency() could drop the last ref" - this last
>> ref is the original refcount set by dma_fence_init->kref
>>
>> Andrey
>
>
> You can see that drm_sched_job_add_dependency() has three return paths
> they are [4], [5] and [1]. [4] and [5] will return 0. [1] will return
> error.
>
> There will be three weird problems if you're right:
>
> 1. [5] path will triger a refcount leak beacause ret is 0 in *if*[6].
Terminology confusion issue - [5] is a 'put' so it cannot cause a leak
by definition, extra unbalanced 'get' will cause a leak because memory
is never released, extra put will just probably cause a warning in
kref_put or maybe double free.
> Otherwise [2] and [5] are matching *get* and *put* in here.
Exactly, they are matching - so until this point all good and no 'leak'
then, no ?
>
> 2. [4] path need a additional dma_fence_get() to adds the fence as a
> job dependency. fence is from obj->resv. Taking msm as an example
> obj->resv is from etnaviv_ioctl_gem_submit()->submit_lookup_objects().
> It is not possible that an object has *refcount == 1* but is
> referenced in two places. So dma_fence_get() called in [2] is for [4].
> By the way, [3] don't execute in this case.
Still don't see the problem - [2] is the additional dma_fence_get() you
need here (just as you say above).
>
> 3. This one is a doubt. You can see in "[PATCH] drm/scheduler: fix
> drm_sched_job_add_implicit_dependencies harder".
> drm_sched_job_add_dependency() could drop the last ref, so we need to do
> the dma_fence_get() first. But the last ref still will drop in [3] if
> drm_sched_job_add_dependency() go path [1]. And there is only a
> *return* between [1] and [3]. Is this necessary? I think Rob Clark
> wants to avoid the last ref being dropped in
> drm_sched_job_add_implicit_dependencies() because fence is still used
> by obj->resv.
In the scenario above - if we go thorough path [1] refcount before [1]
starts is 2 - one from original kref_init and one from [2] and so it's
balanced against 2 puts (one from [1] and one from [3]) so I still don't
see a problem.
I suggest that you give a specific scenario from fence ref-count
perspective that your patch fixes. I might be wrong but unless you give
a specific case where the 'put' in [3] is redundant I just can't see it.
Andrey
>
>
> int drm_sched_job_add_dependency(struct drm_sched_job *job,
> struct dma_fence *fence)
> {
> ...
> xa_for_each(&job->dependencies, index, entry) {
> if (entry->context != fence->context)
> continue;
>
> if (dma_fence_is_later(fence, entry)) {
> dma_fence_put(entry);
> xa_store(&job->dependencies, index, fence,
> GFP_KERNEL); <---- [4]
> } else {
> dma_fence_put(fence); <---- [5]
> }
> return 0;
> }
>
> ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b,
> GFP_KERNEL);
> if (ret != 0)
> dma_fence_put(fence); <---- [1]
>
> return ret;
> }
>
>
> int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> struct drm_gem_object *obj,
> bool write)
> {
> struct dma_resv_iter cursor;
> struct dma_fence *fence;
> int ret;
>
> dma_resv_for_each_fence(&cursor, obj->resv, write, fence) {
> /* Make sure to grab an additional ref on the added
> fence */
> dma_fence_get(fence); <---- [2]
> ret = drm_sched_job_add_dependency(job, fence);
> if (ret) { <---- [6]
> dma_fence_put(fence); <---- [3]
>
> return ret;
> }
> }
> return 0;
> }
>
> Thanks,
> hangyu
>
>>
>>>
>>>
>>> int drm_sched_job_add_dependency(struct drm_sched_job *job,
>>> struct dma_fence *fence)
>>> {
>>> ...
>>> ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b,
>>> GFP_KERNEL);
>>> if (ret != 0)
>>> dma_fence_put(fence); <--- [1]
>>>
>>> return ret;
>>> }
>>> EXPORT_SYMBOL(drm_sched_job_add_dependency);
>>>
>>>
>>> int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
>>> struct drm_gem_object *obj,
>>> bool write)
>>> {
>>> struct dma_resv_iter cursor;
>>> struct dma_fence *fence;
>>> int ret;
>>>
>>> dma_resv_for_each_fence(&cursor, obj->resv, write, fence) {
>>> /* Make sure to grab an additional ref on the added fence */
>>> dma_fence_get(fence); <--- [2]
>>> ret = drm_sched_job_add_dependency(job, fence);
>>> if (ret) {
>>> dma_fence_put(fence); <--- [3]
>>> return ret;
>>> }
>>> }
>>> return 0;
>>> }
>>>
>>>
>>>>
>>>>> On the other hand, dma_fence_get() and dma_fence_put() are
>>>>> meaningless here if threre is an extra dma_fence_get() beacause
>>>>> counter will not decrease to 0 during drm_sched_job_add_dependency().
>>>>>
>>>>> I check the call chain as follows:
>>>>>
>>>>> msm_ioctl_gem_submit()
>>>>> -> submit_fence_sync()
>>>>> -> drm_sched_job_add_implicit_dependencies()
>>>>
>>>>
>>>> Can you maybe trace or print one such example of problematic
>>>> refcount that you are trying to fix ? I still don't see where is
>>>> the problem.
>>>>
>>>> Andrey
>>>>
>>>
>>> I also wish I could. System logs can make this easy. But i don't
>>> have a corresponding GPU physical device.
>>> drm_sched_job_add_implicit_dependencies is only used in a few devices.
>>>
>>> Thanks.
>>>>
>>>>>
>>>>> Thanks,
>>>>> Hangyu
>>>>>
>>>>>>
>>>>>>> return ret;
>>>>>>> }
>>>>>>> }
More information about the dri-devel
mailing list