[PATCH 2/2] drm/vram-helper: Alternate between bottom-up and top-down placement

[Thomas Zimmermann]

Hi Gerd,

thanks for the feedback.

Am 23.04.20 um 13:18 schrieb Gerd Hoffmann:
> On Wed, Apr 22, 2020 at 04:40:55PM +0200, Thomas Zimmermann wrote:
>> With limited VRAM available, fragmentation can lead to OOM errors.
>> Alternating between bottom-up and top-down placement keeps BOs near the
>> ends of the VRAM and the available pages consecutively near the middle.
>>
>> A real-world example with 16 MiB of VRAM is shown below.
>>
>>   > cat /sys/kernel/debug/dri/0/vram-mm
>>   0x0000000000000000-0x000000000000057f: 1407: free
>>   0x000000000000057f-0x0000000000000b5b: 1500: used
>>   0x0000000000000b5b-0x0000000000000ff0: 1173: free
>>
>> The first free area was the location of the fbdev framebuffer. The used
>> area is Weston's current framebuffer of 1500 pages. Weston now cannot
>> do a pageflip to another 1500 page-wide framebuffer, even though enough
>> pages are available. The patch resolves this problem to
>>
>>   > cat /sys/kernel/debug/dri/0/vram-mm
>>   0x0000000000000000-0x00000000000005dc: 1500: used
>>   0x00000000000005dc-0x0000000000000a14: 1080: free
>>   0x0000000000000a14-0x0000000000000ff0: 1500: used
>>
>> with both of Weston's framebuffers located near the ends of the VRAM
>> memory.
> 
> I don't think it is that simple.
> 
> First:  How will that interact with cursor bo allocations?  IIRC the
> strategy for them is to allocate top-down, for similar reasons (avoid
> small cursor bo allocs fragment vram memory).

In ast, 2 cursor BOs are allocated during driver initialization and kept
permanently at the vram's top end. I don't know about other drivers.

But cursor BOs are small, so they don't make much of a difference. What
is needed is space for 2 primary framebuffers during pageflips, with one
of them pinned. The other framebuffer can be located anywhere.


> 
> Second:  I think ttm will move bo's from vram to system only on memory
> pressure.  So you can still end up with fragmented memory.  To make the
> scheme with one fb @ top and one @ bottom work reliable you have to be
> more aggressive on pushing out framebuffers.

I'm the process of converting mgag200 to atomic modesetting. The given
example is what I observed. I'm not claiming that the placement scheme
is perfect, but it is required to get mgag200 working with atomic
modesetting's pageflip logic. So we're solving a real problem here.

The bug comes from Weston's allocation strategy. Looking at the debug
output:

>>   0x0000000000000000-0x000000000000057f: 1407: free

This was fbdev's framebuffer with 1600x900 at 32bpp

>>   0x000000000000057f-0x0000000000000b5b: 1500: used

This is Weston's framebuffer also with 1600x900 at 32bpp. But Weston
allocates an additional, unused 60 scanlines. That is to render with
tiles of 64x64px, I suppose. fbdev doesn't do that, hence Weston's
second framebuffer doesn't fit into the free location of the fbdev
framebuffer.

The other drivers with a small amount of vram are also prone to this
problem. They simply have not yet encountered such a setup.


> 
> Third:  I'd suggest make topdown allocations depending on current state
> instead of simply alternating, i.e. if there is a pinned framebuffer @
> offset 0, then go for top-down.

That's what the current patch does. If the last pin was at the bottom,
the next goes to the top. And then the other way around. Without
alternating between both end of vram, the problem would occur again when
fragmentation happens near the top end.


> 
> I also think using this scheme should be optional.  In the simplest case
> we can allow drivers opt-in.  Or we try do to something clever
> automatically: using the strategy only for framebuffers larger than 1/4
> or 1/3 of total vram memory (which makes alloc failures due to
> fragmentation much more likely).

I'd like to not change behavior automatically, but we can surely make
this optional.

Looking again at the vram helpers, this functionality could be
implemented in drm_gem_vram_plane_helper_prepare_fb(). Drivers with
other placement strategies could implement their own helper for prepare_fb.

Best regards
Thomas


> 
> cheers,
>   Gerd
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/dri-devel/attachments/20200423/e567e344/attachment.sig>