[PATCH 3/9] drm/amdgpu: add amdgpu module parameter for zfb

[Christian König]

Am 08.02.2018 um 05:51 schrieb Zhang, Hawking:
> Apart from the allocation failure, reserving 1024M cma is also not realistic....

Sure it is, I just tried it and it works perfectly fine. The option even 
accepts a G postfix for gigabytes, e.g. I can specify cma=8G on my 32GB 
Ryzen box and get exactly 8GB CMA space reserved.

Regarding the allocation failure using the DMA path there was a thread 
quite some time ago. IIRC the argument was that TTM only allocates a 
single page at a time and so should never run into any CMA allocator and 
according to my tests at least with IOMMU enabled that seems to be fixed 
now.

The key point is that CMA was specially crafted to avoid all those mem= 
workarounds and I think this is now a must have. See here 
https://de.slideshare.net/kerneltlv/continguous-memory-allocator-in-the-linux-kernel 
as well.

Regards,
Christian.

>
> Regards,
> Hawking
>
> -----Original Message-----
> From: amd-gfx [mailto:amd-gfx-bounces at lists.freedesktop.org] On Behalf Of Chunming Zhou
> Sent: Thursday, February 08, 2018 12:35
> To: Koenig, Christian <Christian.Koenig at amd.com>; Xu, Feifei <Feifei.Xu at amd.com>; amd-gfx at lists.freedesktop.org
> Subject: Re: [PATCH 3/9] drm/amdgpu: add amdgpu module parameter for zfb
>
>
>
> On 2018年02月08日 02:48, Christian König wrote:
>> Hi Feifei,
>>
>> ok the correct approach of doing this:
>> 1. Compile the kernel with CONFIG_DMA_CMA.
>> 2. Specify cma=1025M on the kernel command line.
>> 3. Call dma_alloc_coherent() to allocate up to 1GB of memory for ZFB.
> Have you tried CMA? In early time, I tried it. If CMA is enabled, then ttm dma pool allocation will use cma, and when cma is used up, then ttm dma pool allocation will fail.
> Not sure if the issue is fixed now.
>
> Regards,
> David Zhou
>> Couldn't figure out of hand what alignment is necessary here, but that
>> should just be a typing exercise.
>>
>> Regards,
>> Christian.
>>
>> Am 07.02.2018 um 14:18 schrieb Christian König:
>>> Hi Feifei,
>>>
>>> mhm, so you're approach is to limit the memory the Linux kernel grab
>>> using the mem= parameter and then give that memory to amdgpu, correct?
>>>
>>> If that's the case then this is most likely a NAK. Cause this sounds
>>> like a perfect example for the CMA.
>>>
>>> Never worked with that subsystem before, but in theory you should be
>>> able to give the cma=1G parameter to the kernel and by doing so you
>>> reserve 1GB of contiguous memory for use by device drivers.
>>>
>>> Going to dig a bit deeper into that and trying to find to
>>> documentation about it, Christian.
>>>
>>> Am 07.02.2018 um 13:34 schrieb Feifei Xu:
>>>> Users can pass in an array to decide enable/disable Zero Frame Buffer.
>>>> zfb[0] = zfb_size(MB), zfb[1] = zfb_phys_addr(MB).
>>>> If zbf_size > 0, zfb is enabled. Otherwise disabled.
>>>> Usage for example:
>>>>       modprobe amdgpu zfb={256,4096}
>>>>
>>>> Change-Id: I340fcb36b5655f24551056e685b74559d7599680
>>>> Signed-off-by: Feifei Xu <Feifei.Xu at amd.com>
>>>> ---
>>>>    drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
>>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 5 +++++
>>>>    2 files changed, 6 insertions(+)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> index 1b4c5ad..be79cb1 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> @@ -129,6 +129,7 @@ extern int amdgpu_job_hang_limit;
>>>>    extern int amdgpu_lbpw;
>>>>    extern int amdgpu_compute_multipipe;
>>>>    extern int amdgpu_gpu_recovery;
>>>> +extern ulong amdgpu_zfb[];
>>>>      #ifdef CONFIG_DRM_AMDGPU_SI
>>>>    extern int amdgpu_si_support;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> index 3897179..af84815 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> @@ -132,6 +132,7 @@ int amdgpu_job_hang_limit = 0;
>>>>    int amdgpu_lbpw = -1;
>>>>    int amdgpu_compute_multipipe = -1;
>>>>    int amdgpu_gpu_recovery = -1; /* auto */
>>>> +ulong amdgpu_zfb[2] = {0,4096UL}; /* {0,0x100000000} */
>>>>      MODULE_PARM_DESC(vramlimit, "Restrict VRAM for testing, in
>>>> megabytes");
>>>>    module_param_named(vramlimit, amdgpu_vram_limit, int, 0600); @@
>>>> -290,6 +291,10 @@ module_param_named(compute_multipipe,
>>>> amdgpu_compute_multipipe, int, 0444);
>>>>    MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (1
>>>> = enable, 0 = disable, -1 = auto");
>>>>    module_param_named(gpu_recovery, amdgpu_gpu_recovery, int, 0444);
>>>>    +MODULE_PARM_DESC(zfb,
>>>> +         "Enable Zero Frame Buffer feature (zfb will be set like
>>>> xxxx,xxxx(zfb_size MB,zfb_phys_addr MB),default disabled)");
>>>> +module_param_array_named(zfb, amdgpu_zfb, ulong, NULL, 0444);
>>>> +
>>>>    #ifdef CONFIG_DRM_AMDGPU_SI
>>>>      int amdgpu_si_support = 1;
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx