[PATCH 3/9] drm/amdgpu: add amdgpu module parameter for zfb
He, Roger
Hongbo.He at amd.com
Thu Feb 8 08:40:44 UTC 2018
For the CMA, I have some concern.
First, it is strongly depend on other module's robustness which can use CMA memory with flag __GFP_MOVABLE.
My mean is dma_alloc_coherent sometimes will fail because its owner cannot migrate its memory out from CMA range because some internal bugs.
Second because memory migration, the CMA memory allocation performance is not very good as normal.
When I used CMA feature with 3.16 kernel encountered more issues.
So not pretty sure CMA module and other modules are very robust in 4.15 kernel.
Thanks
Roger(Hongbo.He)
-----Original Message-----
From: amd-gfx [mailto:amd-gfx-bounces at lists.freedesktop.org] On Behalf Of Christian K?nig
Sent: Thursday, February 08, 2018 4:10 PM
To: Zhang, Hawking <Hawking.Zhang at amd.com>; Zhou, David(ChunMing) <David1.Zhou at amd.com>; Xu, Feifei <Feifei.Xu at amd.com>; amd-gfx at lists.freedesktop.org
Subject: Re: [PATCH 3/9] drm/amdgpu: add amdgpu module parameter for zfb
Am 08.02.2018 um 05:51 schrieb Zhang, Hawking:
> Apart from the allocation failure, reserving 1024M cma is also not realistic....
Sure it is, I just tried it and it works perfectly fine. The option even accepts a G postfix for gigabytes, e.g. I can specify cma=8G on my 32GB Ryzen box and get exactly 8GB CMA space reserved.
Regarding the allocation failure using the DMA path there was a thread quite some time ago. IIRC the argument was that TTM only allocates a single page at a time and so should never run into any CMA allocator and according to my tests at least with IOMMU enabled that seems to be fixed now.
The key point is that CMA was specially crafted to avoid all those mem= workarounds and I think this is now a must have. See here https://de.slideshare.net/kerneltlv/continguous-memory-allocator-in-the-linux-kernel
as well.
Regards,
Christian.
>
> Regards,
> Hawking
>
> -----Original Message-----
> From: amd-gfx [mailto:amd-gfx-bounces at lists.freedesktop.org] On Behalf
> Of Chunming Zhou
> Sent: Thursday, February 08, 2018 12:35
> To: Koenig, Christian <Christian.Koenig at amd.com>; Xu, Feifei
> <Feifei.Xu at amd.com>; amd-gfx at lists.freedesktop.org
> Subject: Re: [PATCH 3/9] drm/amdgpu: add amdgpu module parameter for
> zfb
>
>
>
> On 2018年02月08日 02:48, Christian König wrote:
>> Hi Feifei,
>>
>> ok the correct approach of doing this:
>> 1. Compile the kernel with CONFIG_DMA_CMA.
>> 2. Specify cma=1025M on the kernel command line.
>> 3. Call dma_alloc_coherent() to allocate up to 1GB of memory for ZFB.
> Have you tried CMA? In early time, I tried it. If CMA is enabled, then ttm dma pool allocation will use cma, and when cma is used up, then ttm dma pool allocation will fail.
> Not sure if the issue is fixed now.
>
> Regards,
> David Zhou
>> Couldn't figure out of hand what alignment is necessary here, but
>> that should just be a typing exercise.
>>
>> Regards,
>> Christian.
>>
>> Am 07.02.2018 um 14:18 schrieb Christian König:
>>> Hi Feifei,
>>>
>>> mhm, so you're approach is to limit the memory the Linux kernel grab
>>> using the mem= parameter and then give that memory to amdgpu, correct?
>>>
>>> If that's the case then this is most likely a NAK. Cause this sounds
>>> like a perfect example for the CMA.
>>>
>>> Never worked with that subsystem before, but in theory you should be
>>> able to give the cma=1G parameter to the kernel and by doing so you
>>> reserve 1GB of contiguous memory for use by device drivers.
>>>
>>> Going to dig a bit deeper into that and trying to find to
>>> documentation about it, Christian.
>>>
>>> Am 07.02.2018 um 13:34 schrieb Feifei Xu:
>>>> Users can pass in an array to decide enable/disable Zero Frame Buffer.
>>>> zfb[0] = zfb_size(MB), zfb[1] = zfb_phys_addr(MB).
>>>> If zbf_size > 0, zfb is enabled. Otherwise disabled.
>>>> Usage for example:
>>>> modprobe amdgpu zfb={256,4096}
>>>>
>>>> Change-Id: I340fcb36b5655f24551056e685b74559d7599680
>>>> Signed-off-by: Feifei Xu <Feifei.Xu at amd.com>
>>>> ---
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 5 +++++
>>>> 2 files changed, 6 insertions(+)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> index 1b4c5ad..be79cb1 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> @@ -129,6 +129,7 @@ extern int amdgpu_job_hang_limit;
>>>> extern int amdgpu_lbpw;
>>>> extern int amdgpu_compute_multipipe;
>>>> extern int amdgpu_gpu_recovery;
>>>> +extern ulong amdgpu_zfb[];
>>>> #ifdef CONFIG_DRM_AMDGPU_SI
>>>> extern int amdgpu_si_support;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> index 3897179..af84815 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>>> @@ -132,6 +132,7 @@ int amdgpu_job_hang_limit = 0;
>>>> int amdgpu_lbpw = -1;
>>>> int amdgpu_compute_multipipe = -1;
>>>> int amdgpu_gpu_recovery = -1; /* auto */
>>>> +ulong amdgpu_zfb[2] = {0,4096UL}; /* {0,0x100000000} */
>>>> MODULE_PARM_DESC(vramlimit, "Restrict VRAM for testing, in
>>>> megabytes");
>>>> module_param_named(vramlimit, amdgpu_vram_limit, int, 0600); @@
>>>> -290,6 +291,10 @@ module_param_named(compute_multipipe,
>>>> amdgpu_compute_multipipe, int, 0444);
>>>> MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism,
>>>> (1 = enable, 0 = disable, -1 = auto");
>>>> module_param_named(gpu_recovery, amdgpu_gpu_recovery, int,
>>>> 0444);
>>>> +MODULE_PARM_DESC(zfb,
>>>> + "Enable Zero Frame Buffer feature (zfb will be set like
>>>> xxxx,xxxx(zfb_size MB,zfb_phys_addr MB),default disabled)");
>>>> +module_param_array_named(zfb, amdgpu_zfb, ulong, NULL, 0444);
>>>> +
>>>> #ifdef CONFIG_DRM_AMDGPU_SI
>>>> int amdgpu_si_support = 1;
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
More information about the amd-gfx
mailing list