vc4_bo_create: Failed to allocate from CMA

[nicolas saenz julienne]

On Sat, 2021-06-12 at 17:17 +0200, Stefan Wahren wrote:
> Hi,
> 
> while testing the mainline kernel (arm64, defconfig) on Raspberry Pi 3 B
> Plus with Raspberry Pi OS - 64 bit, sometimes X doesn't start into
> desktop properly (unexpected and unusable login screen instead of auto
> login or mouse pointer is show shorty and than switch back to black
> screen in a loop). In that case dmesg shows the following:
> 
> [   74.737106] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
> CMA:
> [   74.737558] vc4-drm soc:gpu: [drm]                            V3D: 
> 28976kb BOs (10)
> [   74.737602] vc4-drm soc:gpu: [drm]                     V3D
> shader:     44kb BOs (11)
> [   74.737632] vc4-drm soc:gpu: [drm]                           dumb:  
> 4564kb BOs (5)
> [   74.737664] vc4-drm soc:gpu: [drm]                         binner: 
> 16384kb BOs (1)
> [   74.737697] vc4-drm soc:gpu: [drm]                total purged
> BO:      4kb BOs (1)
> [   74.739039] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
> CMA:
> [   74.739466] vc4-drm soc:gpu: [drm]                            V3D: 
> 28972kb BOs (9)
> [   74.739512] vc4-drm soc:gpu: [drm]                     V3D
> shader:     44kb BOs (11)
> [   74.739541] vc4-drm soc:gpu: [drm]                           dumb:  
> 4564kb BOs (5)
> [   74.739570] vc4-drm soc:gpu: [drm]                         binner: 
> 16384kb BOs (1)
> [   74.739602] vc4-drm soc:gpu: [drm]                total purged
> BO:      4kb BOs (1)
> [   74.740718] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
> CMA:
> [   74.741138] vc4-drm soc:gpu: [drm]                            V3D: 
> 28972kb BOs (9)
> [   74.741171] vc4-drm soc:gpu: [drm]                     V3D
> shader:     44kb BOs (11)
> [   74.741202] vc4-drm soc:gpu: [drm]                           dumb:  
> 4564kb BOs (5)
> [   74.741231] vc4-drm soc:gpu: [drm]                         binner: 
> 16384kb BOs (1)
> [   74.741263] vc4-drm soc:gpu: [drm]                total purged
> BO:      4kb BOs (1)
> ...
> 
> I have only seen this issue on arm64 with latest mainline kernel
> (5.13.0-rc5-00130-gf21b807c3cf8), but also with older kernel versions.
> So it's not a regression. It seems 64 bit needs more CMA.
> 
> In case X started properly i was also able to reproduce these errors
> above by dis- and reconneting HDMI.
> 
> So i increased CMA in bcm283x.dtsi and the problem disappeared:
> 
> iff --git a/arch/arm/boot/dts/bcm283x.dtsi b/arch/arm/boot/dts/bcm283x.dtsi
> index b83a864..d1304cb 100644
> --- a/arch/arm/boot/dts/bcm283x.dtsi
> +++ b/arch/arm/boot/dts/bcm283x.dtsi
> @@ -37,7 +37,7 @@
>  
>          cma: linux,cma {
>              compatible = "shared-dma-pool";
> -            size = <0x4000000>; /* 64MB */
> +            size = <0x6000000>; /* 96MB */
>              reusable;
>              linux,cma-default;
>          };
> 
> The questions are:
> 
> Is this the right way (tm) to fix this problem?

Frankly I don't know if there is a better way. IIRC opensuse and downstream use
DT overlays to cater for this limitation. It seems reasonable to bump the
value. But it'll be in detriment of users that don't care much for graphical
interfaces. Nonetheless, I'm not familiar with how DRM handles CMA/DMA memory.
So let me have a look at it. Maybe there is a SW fix. At first glance I'm
surprised they can't defer to normal page allocations when CMA isn't capable of
honoring the request (like the dma code does).

> And what is a sensible value (don't have a 4K display to test)?

The default for downstream is 256MB. But I've read discussions in the forum
where people needed even more. IIUC it's use-case dependent, resolution is only
one variable, you might then try to run a game and run out of memory there.

Regards,
Nicolas