Amdgpu module is references even after unbinding the vtcon

[Christian König]

Oh, yeah. Very good point as well.

Christian.

Am 26.01.23 um 15:13 schrieb Sharma, Shashank:
> [AMD Official Use Only - General]
>
> I would also highly recommend this to be tested with another compositor (Like Weston/Sway etc)
>
> Regards
> Shashank
> -----Original Message-----
> From: Christian König <ckoenig.leichtzumerken at gmail.com>
> Sent: 26 January 2023 15:12
> To: Slivka, Danijel <Danijel.Slivka at amd.com>; Thomas Zimmermann <tzimmermann at suse.de>
> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; dri-devel <dri-devel at lists.freedesktop.org>; Sharma, Shashank <Shashank.Sharma at amd.com>
> Subject: Re: Amdgpu module is references even after unbinding the vtcon
>
> Hi Danijel,
>
> can you also double check that GDM/X is still capable of acquiring a DMA-buf file descriptor for the buffer (e.g. that we have a DMA-buf handle for it while they are started).
>
> And that handover from fbdev to GDM/X is flicker free?
>
> Thanks,
> Christian.
>
> Am 26.01.23 um 14:44 schrieb Slivka, Danijel:
>> [AMD Official Use Only - General]
>>
>> Hi Christian,
>>
>> I have tested the proposed patch, the issue is not reproducible and everything else seems to work fine.
>>
>> BR,
>> Danijel
>>
>>> -----Original Message-----
>>> From: Christian König <ckoenig.leichtzumerken at gmail.com>
>>> Sent: Thursday, January 26, 2023 1:20 PM
>>> To: Slivka, Danijel <Danijel.Slivka at amd.com>; Thomas Zimmermann
>>> <tzimmermann at suse.de>
>>> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; dri-devel <dri-
>>> devel at lists.freedesktop.org>; Sharma, Shashank
>>> <Shashank.Sharma at amd.com>
>>> Subject: Re: Amdgpu module is references even after unbinding the
>>> vtcon
>>>
>>> Am 26.01.23 um 10:49 schrieb Slivka, Danijel:
>>>> [AMD Official Use Only - General]
>>>>
>>>> Hi Thomas,
>>>>
>>>> I have checked what you mentioned.
>>>> When loading amdgpu we call  drm_client_init() during fbdev setup
>>>> [1], the
>>> refcnt for drm_kms_helper increases from 3 -> 4.
>>>> When we unbind vtcon, refcnt for drm_kms_helper drops 4 -> 3, but
>>>> the
>>> drm_client_release() [2] is not called.
>>>> The drm_client_release() is called only when unloading the amdgpu driver.
>>>>
>>>> Is this expected?
>>> Yes, the client can't be released because it is possible that the
>>> vtcon is bound to this fbdev again.
>>>
>>> Please test the handle work around I've send around internally. At
>>> least for me that approach seems to work.
>>>
>>> Regards,
>>> Christian.
>>>
>>>> There is a comment for drm_client_release with regards to fbdev :
>>>> * This function should only be called from the unregister callback. An exception
>>>>     * is fbdev which cannot free the buffer if userspace has open file descriptors.
>>>>
>>>> Could this be relevant for our use case, although as
>>>> Application/X/GDM are
>>> stopped at that point and no fd should be open.
>>>> Thank you,
>>>> BR,
>>>> Danijel
>>>>
>>>>> -----Original Message-----
>>>>> From: Thomas Zimmermann <tzimmermann at suse.de>
>>>>> Sent: Wednesday, January 25, 2023 8:48 PM
>>>>> To: Christian König <ckoenig.leichtzumerken at gmail.com>
>>>>> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Slivka, Danijel
>>>>> <Danijel.Slivka at amd.com>; dri-devel
>>>>> <dri-devel at lists.freedesktop.org>; Sharma, Shashank
>>>>> <Shashank.Sharma at amd.com>
>>>>> Subject: Re: Amdgpu module is references even after unbinding the
>>>>> vtcon
>>>>>
>>>>> Hi Christian
>>>>>
>>>>> Am 24.01.23 um 15:12 schrieb Christian König:
>>>>>> Hi Thomas,
>>>>>>
>>>>>> we ran into a problem with the general fbcon/fbdev implementation
>>>>>> and though that you might have some idea.
>>>>>>
>>>>>> What happens is the following:
>>>>>> 1. We load amdgpu and get our normal fbcon.
>>>>>> 2. fbcon allocates a dump BO as backing store for the console.
>>>>>> 3. GDM/X/Applications start, new framebuffers are created BOs
>>>>>> imported, exported etc...
>>>>>> 4. Somehow X or GDM iterated over all the framebuffer objects the
>>>>>> kernels knows about and export them as DMA-buf.
>>>>>> 5. Application/X/GDM are stopped, handles closed, framebuffers
>>>>>> released etc...
>>>>>> 6. We unbind vtcon.
>>>>>>
>>>>>> At this point the amdgpu module usually has a reference count of 0
>>>>>> and can be unloaded, but since GDM/X/Whoever iterated over all the
>>>>>> known framebuffers and exported them as DMA-buf (for whatever
>>>>>> reason
>>>>>> idk) we now still have an exported DMA-buf and with it a reference
>>>>>> to the
>>> module.
>>>>>> Any idea how we could prevent that?
>>>>> Here's another stab in the dark.
>>>>>
>>>>> The big difference between old-style fbdev and the new one is that
>>>>> the old fbdev setup (e.g., radeon) allocates a GEM object and puts
>>>>> together the fbdev data structures from the BO in a fairly hackish
>>>>> way. The new style uses an in-kernel client with a file to allocate
>>>>> the BO via dumb buffers; and holds a reference to the DRM module.
>>>>>
>>>>> Maybe the reference comes from the in-kernel DRM client itself. [1]
>>>>> Check if the client resources get released [2] when you unbind vtcon.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1]
>>>>> https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_
>>>>> cl
>>>>> ient.c#L87
>>>>> [2]
>>>>> https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_
>>>>> cl
>>>>> ient.c#L16
>>>>> 0
>>>>>
>>>>>> Thanks,
>>>>>> Christian.
>>>>> --
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg,
>>>>> Germany (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Ivo Totev