vc4_bo_create: Failed to allocate from CMA

[Stefan Wahren]

Hi Nicolas,

Am 17.06.21 um 11:36 schrieb nicolas saenz julienne:
> On Sat, 2021-06-12 at 17:17 +0200, Stefan Wahren wrote:
>> Hi,
>>
>> while testing the mainline kernel (arm64, defconfig) on Raspberry Pi 3 B
>> Plus with Raspberry Pi OS - 64 bit, sometimes X doesn't start into
>> desktop properly (unexpected and unusable login screen instead of auto
>> login or mouse pointer is show shorty and than switch back to black
>> screen in a loop). In that case dmesg shows the following:
>>
>> [   74.737106] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
>> CMA:
>> [   74.737558] vc4-drm soc:gpu: [drm]                            V3D: 
>> 28976kb BOs (10)
>> [   74.737602] vc4-drm soc:gpu: [drm]                     V3D
>> shader:     44kb BOs (11)
>> [   74.737632] vc4-drm soc:gpu: [drm]                           dumb:  
>> 4564kb BOs (5)
>> [   74.737664] vc4-drm soc:gpu: [drm]                         binner: 
>> 16384kb BOs (1)
>> [   74.737697] vc4-drm soc:gpu: [drm]                total purged
>> BO:      4kb BOs (1)
>> [   74.739039] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
>> CMA:
>> [   74.739466] vc4-drm soc:gpu: [drm]                            V3D: 
>> 28972kb BOs (9)
>> [   74.739512] vc4-drm soc:gpu: [drm]                     V3D
>> shader:     44kb BOs (11)
>> [   74.739541] vc4-drm soc:gpu: [drm]                           dumb:  
>> 4564kb BOs (5)
>> [   74.739570] vc4-drm soc:gpu: [drm]                         binner: 
>> 16384kb BOs (1)
>> [   74.739602] vc4-drm soc:gpu: [drm]                total purged
>> BO:      4kb BOs (1)
>> [   74.740718] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from
>> CMA:
>> [   74.741138] vc4-drm soc:gpu: [drm]                            V3D: 
>> 28972kb BOs (9)
>> [   74.741171] vc4-drm soc:gpu: [drm]                     V3D
>> shader:     44kb BOs (11)
>> [   74.741202] vc4-drm soc:gpu: [drm]                           dumb:  
>> 4564kb BOs (5)
>> [   74.741231] vc4-drm soc:gpu: [drm]                         binner: 
>> 16384kb BOs (1)
>> [   74.741263] vc4-drm soc:gpu: [drm]                total purged
>> BO:      4kb BOs (1)
>> ...
>>
>> I have only seen this issue on arm64 with latest mainline kernel
>> (5.13.0-rc5-00130-gf21b807c3cf8), but also with older kernel versions.
>> So it's not a regression. It seems 64 bit needs more CMA.
>>
>> In case X started properly i was also able to reproduce these errors
>> above by dis- and reconneting HDMI.
>>
>> So i increased CMA in bcm283x.dtsi and the problem disappeared:
>>
>> iff --git a/arch/arm/boot/dts/bcm283x.dtsi b/arch/arm/boot/dts/bcm283x.dtsi
>> index b83a864..d1304cb 100644
>> --- a/arch/arm/boot/dts/bcm283x.dtsi
>> +++ b/arch/arm/boot/dts/bcm283x.dtsi
>> @@ -37,7 +37,7 @@
>>  
>>          cma: linux,cma {
>>              compatible = "shared-dma-pool";
>> -            size = <0x4000000>; /* 64MB */
>> +            size = <0x6000000>; /* 96MB */
>>              reusable;
>>              linux,cma-default;
>>          };
>>
>> The questions are:
>>
>> Is this the right way (tm) to fix this problem?
> Frankly I don't know if there is a better way. IIRC opensuse and downstream use
> DT overlays to cater for this limitation. It seems reasonable to bump the
> value. But it'll be in detriment of users that don't care much for graphical
> interfaces. Nonetheless, I'm not familiar with how DRM handles CMA/DMA memory.
> So let me have a look at it. Maybe there is a SW fix. At first glance I'm
> surprised they can't defer to normal page allocations when CMA isn't capable of
> honoring the request (like the dma code does).

a compromise might be to increase the CMA size based on the SoC type
(newer generations have more memory)

BCM2835 => 64 MB
BCM2836, BCM2837 => 256 MB

>
>> And what is a sensible value (don't have a 4K display to test)?
> The default for downstream is 256MB. But I've read discussions in the forum
> where people needed even more. IIUC it's use-case dependent, resolution is only
> one variable, you might then try to run a game and run out of memory there.

Sure this wasn't intend to make everybody happy. But i would expect to
start X reliable at least.

Regards
Stefan

>
> Regards,
> Nicolas
>