[RFC PATCH] drm/prime: introduce DRM_PRIME_FD_TO_HANDLE_NO_MOVE

[Michel Dänzer]

On 2024-10-15 13:01, Simon Ser wrote:
> On Tuesday, October 15th, 2024 at 12:47, Michel Dänzer <michel.daenzer at mailbox.org> wrote:
> 
>> On 2024-10-13 15:34, Simon Ser wrote:
>>
>>> This is a flag to opt-out of the automagic buffer migration to
>>> system memory when importing a DMA-BUF.
>>>
>>> In multi-GPU scenarii, a Wayland client might allocate on any
>>> device. The Wayland compositor receiving the DMA-BUF has no clue
>>> where the buffer has been allocated from. The compositor will
>>> typically try to import the buffer into its "primary" device,
>>> although it would be capable of importing into any DRM device.
>>>
>>> This causes issues in case buffer imports implicitly result in
>>> the buffer being moved to system memory. For instance, on a
>>> system with an Intel iGPU and an AMD dGPU, a client rendering
>>> with the dGPU and whose window is displayed on a screen
>>> connected to the dGPU would ideally not need any roundtrip
>>> to the iGPU. However, any attempt at figuring out where the
>>> DMA-BUF could be accessed from will move the buffer into system
>>> memory, degrading performance for the rest of the lifetime of the
>>> buffer.
>>>
>>> Describing on which device the buffer has been allocated on is
>>> not enough: on some setups the buffer may have been allocated on
>>> one device but may still be directly accessible without any move
>>> on another device. For instance, on a split render/display system,
>>> a buffer allocated on the display device can be directly rendered
>>> to from the render device.
>>>
>>> With this new flag, a compositor can try to import on all DRM
>>> devices without any side effects. If it finds a device which can
>>> access the buffer without a move, it can use that device to render
>>> the buffer. If it doesn't, it can fallback to the previous
>>> behavior: try to import without the flag to the "primary" device,
>>> knowing this could result in a move to system memory.
>>
>> One problem with this approach is that even if a buffer is originally created in / intended for local VRAM of a dGPU, it may get temporarily migrated to system RAM for other reasons, e.g. to make room for other buffers in VRAM. While it resides in system RAM, importing into another device with DRM_PRIME_FD_TO_HANDLE_NO_MOVE will work, but will result in pinning the buffer to system RAM, even though this isn't optimal for the intended buffer usage.
> 
> Indeed. Do you think we could have a flag which also prevents pinning?
> 
> Sounds like that would involve implementing dynamic DMA-BUF importers in
> GEM? (Some drivers like xe already implement that.)

As discussed on IRC, that won't always help for the case I described. The exporting GPU will want to migrate the BO to its VRAM for drawing to it, the importing device might only be able to access it in system RAM though. The result would be migration ping-pong of the BO between VRAM and system RAM.


Taking a step back, I suspect something like "importing device can access while exporting device has optimal access" might be more useful than "can import without moving". Though that would still leave unknown if the importing device's access is optimal or not.


-- 
Earthling Michel Dänzer       \        GNOME / Xwayland / Mesa developer
https://redhat.com             \               Libre software enthusiast