Amdgpu module is references even after unbinding the vtcon

[Slivka, Danijel]

[AMD Official Use Only - General]

Hi Christian,

I was not able to check about the handover from fbdev to gdm as I was having issues with setting it up proper driver in the environment with display.
Regarding exporting DMA-buf, when the X starts it acquires file descriptor for the buffer with creating new handle and when closing the file  descriptor it successfully releases the dma buffer that was exported as it is tha last handle on the underlying object.

BR,
Danijel

>-----Original Message-----
>From: Christian König <ckoenig.leichtzumerken at gmail.com>
>Sent: Thursday, January 26, 2023 3:12 PM
>To: Slivka, Danijel <Danijel.Slivka at amd.com>; Thomas Zimmermann
><tzimmermann at suse.de>
>Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; dri-devel <dri-
>devel at lists.freedesktop.org>; Sharma, Shashank
><Shashank.Sharma at amd.com>
>Subject: Re: Amdgpu module is references even after unbinding the vtcon
>
>Hi Danijel,
>
>can you also double check that GDM/X is still capable of acquiring a DMA-buf
>file descriptor for the buffer (e.g. that we have a DMA-buf handle for it while
>they are started).
>
>And that handover from fbdev to GDM/X is flicker free?
>
>Thanks,
>Christian.
>
>Am 26.01.23 um 14:44 schrieb Slivka, Danijel:
>> [AMD Official Use Only - General]
>>
>> Hi Christian,
>>
>> I have tested the proposed patch, the issue is not reproducible and
>everything else seems to work fine.
>>
>> BR,
>> Danijel
>>
>>> -----Original Message-----
>>> From: Christian König <ckoenig.leichtzumerken at gmail.com>
>>> Sent: Thursday, January 26, 2023 1:20 PM
>>> To: Slivka, Danijel <Danijel.Slivka at amd.com>; Thomas Zimmermann
>>> <tzimmermann at suse.de>
>>> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; dri-devel <dri-
>>> devel at lists.freedesktop.org>; Sharma, Shashank
>>> <Shashank.Sharma at amd.com>
>>> Subject: Re: Amdgpu module is references even after unbinding the
>>> vtcon
>>>
>>> Am 26.01.23 um 10:49 schrieb Slivka, Danijel:
>>>> [AMD Official Use Only - General]
>>>>
>>>> Hi Thomas,
>>>>
>>>> I have checked what you mentioned.
>>>> When loading amdgpu we call  drm_client_init() during fbdev setup
>>>> [1], the
>>> refcnt for drm_kms_helper increases from 3 -> 4.
>>>> When we unbind vtcon, refcnt for drm_kms_helper drops 4 -> 3, but
>>>> the
>>> drm_client_release() [2] is not called.
>>>> The drm_client_release() is called only when unloading the amdgpu
>driver.
>>>>
>>>> Is this expected?
>>> Yes, the client can't be released because it is possible that the
>>> vtcon is bound to this fbdev again.
>>>
>>> Please test the handle work around I've send around internally. At
>>> least for me that approach seems to work.
>>>
>>> Regards,
>>> Christian.
>>>
>>>> There is a comment for drm_client_release with regards to fbdev :
>>>> * This function should only be called from the unregister callback. An
>exception
>>>>    * is fbdev which cannot free the buffer if userspace has open file
>descriptors.
>>>>
>>>> Could this be relevant for our use case, although as
>>>> Application/X/GDM are
>>> stopped at that point and no fd should be open.
>>>> Thank you,
>>>> BR,
>>>> Danijel
>>>>
>>>>> -----Original Message-----
>>>>> From: Thomas Zimmermann <tzimmermann at suse.de>
>>>>> Sent: Wednesday, January 25, 2023 8:48 PM
>>>>> To: Christian König <ckoenig.leichtzumerken at gmail.com>
>>>>> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Slivka, Danijel
>>>>> <Danijel.Slivka at amd.com>; dri-devel
>>>>> <dri-devel at lists.freedesktop.org>; Sharma, Shashank
>>>>> <Shashank.Sharma at amd.com>
>>>>> Subject: Re: Amdgpu module is references even after unbinding the
>>>>> vtcon
>>>>>
>>>>> Hi Christian
>>>>>
>>>>> Am 24.01.23 um 15:12 schrieb Christian König:
>>>>>> Hi Thomas,
>>>>>>
>>>>>> we ran into a problem with the general fbcon/fbdev implementation
>>>>>> and though that you might have some idea.
>>>>>>
>>>>>> What happens is the following:
>>>>>> 1. We load amdgpu and get our normal fbcon.
>>>>>> 2. fbcon allocates a dump BO as backing store for the console.
>>>>>> 3. GDM/X/Applications start, new framebuffers are created BOs
>>>>>> imported, exported etc...
>>>>>> 4. Somehow X or GDM iterated over all the framebuffer objects the
>>>>>> kernels knows about and export them as DMA-buf.
>>>>>> 5. Application/X/GDM are stopped, handles closed, framebuffers
>>>>>> released etc...
>>>>>> 6. We unbind vtcon.
>>>>>>
>>>>>> At this point the amdgpu module usually has a reference count of 0
>>>>>> and can be unloaded, but since GDM/X/Whoever iterated over all the
>>>>>> known framebuffers and exported them as DMA-buf (for whatever
>>>>>> reason
>>>>>> idk) we now still have an exported DMA-buf and with it a reference
>>>>>> to the
>>> module.
>>>>>> Any idea how we could prevent that?
>>>>> Here's another stab in the dark.
>>>>>
>>>>> The big difference between old-style fbdev and the new one is that
>>>>> the old fbdev setup (e.g., radeon) allocates a GEM object and puts
>>>>> together the fbdev data structures from the BO in a fairly hackish
>>>>> way. The new style uses an in-kernel client with a file to allocate
>>>>> the BO via dumb buffers; and holds a reference to the DRM module.
>>>>>
>>>>> Maybe the reference comes from the in-kernel DRM client itself. [1]
>>>>> Check if the client resources get released [2] when you unbind vtcon.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1]
>>>>> https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_
>>>>> cl
>>>>> ient.c#L87
>>>>> [2]
>>>>> https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_
>>>>> cl
>>>>> ient.c#L16
>>>>> 0
>>>>>
>>>>>> Thanks,
>>>>>> Christian.
>>>>> --
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409
>Nürnberg,
>>>>> Germany (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Ivo Totev