[PATCH v3 2/2] drm/amdgpu: Put page tables to GTT memory for gfx10 onwards APUs

[Zhang, Yifan]

[AMD Official Use Only - General]

Validated on Phoenix and Raphael w/ IOMMU remapping mode, and found random page fault when launching Xorg. I will debug this issue and update the patch.

Best Regards,
Yifan

From: Deucher, Alexander <Alexander.Deucher at amd.com>
Sent: Tuesday, August 29, 2023 2:06 AM
To: Koenig, Christian <Christian.Koenig at amd.com>; Zhang, Yifan <Yifan1.Zhang at amd.com>; Christian König <ckoenig.leichtzumerken at gmail.com>; amd-gfx at lists.freedesktop.org
Subject: Re: [PATCH v3 2/2] drm/amdgpu: Put page tables to GTT memory for gfx10 onwards APUs


[AMD Official Use Only - General]

Technically the AMD IOMMU uses direct mapping mode for any device which claims to support ATS in order to support the IOMMUv2 functionality, but that was also the case with Raven systems which were problematic when remapping mode was enabled.  That said, now that IOMMUv2 support has been removed, there's no reason to use direct mapping in the IOMMU driver so that may change.

Alex

________________________________
From: Koenig, Christian <Christian.Koenig at amd.com<mailto:Christian.Koenig at amd.com>>
Sent: Monday, August 28, 2023 7:30 AM
To: Zhang, Yifan <Yifan1.Zhang at amd.com<mailto:Yifan1.Zhang at amd.com>>; Christian König <ckoenig.leichtzumerken at gmail.com<mailto:ckoenig.leichtzumerken at gmail.com>>; amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org> <amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>>
Cc: Deucher, Alexander <Alexander.Deucher at amd.com<mailto:Alexander.Deucher at amd.com>>
Subject: Re: [PATCH v3 2/2] drm/amdgpu: Put page tables to GTT memory for gfx10 onwards APUs

Well, there seems to be a very basic misunderstood here: The IOMMU
isolation level is *not* ASIC dependent!

Try to set amd_iommu=force_isolation on the kernel command line.

This is a configuration option customers can use to harden their systems
and when this isn't properly tested we can't allow page tables in system
memory.

Regards,
Christian.

Am 28.08.23 um 13:23 schrieb Zhang, Yifan:
> [Public]
>
> Not yet. It will be only enabled for gfx10.3.3 and later APU initially, IOMMU is pass through in these ASIC.
>
> -----Original Message-----
> From: Christian König <ckoenig.leichtzumerken at gmail.com<mailto:ckoenig.leichtzumerken at gmail.com>>
> Sent: Monday, August 28, 2023 5:41 PM
> To: Zhang, Yifan <Yifan1.Zhang at amd.com<mailto:Yifan1.Zhang at amd.com>>; amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>
> Cc: Deucher, Alexander <Alexander.Deucher at amd.com<mailto:Alexander.Deucher at amd.com>>; Koenig, Christian <Christian.Koenig at amd.com<mailto:Christian.Koenig at amd.com>>
> Subject: Re: [PATCH v3 2/2] drm/amdgpu: Put page tables to GTT memory for gfx10 onwards APUs
>
> Is that now validated with IOMMU in non pass through mode?
>
> Christian.
>
> Am 28.08.23 um 10:58 schrieb Zhang, Yifan:
>> [AMD Official Use Only - General]
>>
>> Ping
>>
>> -----Original Message-----
>> From: Zhang, Yifan <Yifan1.Zhang at amd.com<mailto:Yifan1.Zhang at amd.com>>
>> Sent: Friday, August 25, 2023 8:34 AM
>> To: amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>
>> Cc: Deucher, Alexander <Alexander.Deucher at amd.com<mailto:Alexander.Deucher at amd.com>>; Koenig, Christian <Christian.Koenig at amd.com<mailto:Christian.Koenig at amd.com>>; Zhang, Yifan <Yifan1.Zhang at amd.com<mailto:Yifan1.Zhang at amd.com>>
>> Subject: [PATCH v3 2/2] drm/amdgpu: Put page tables to GTT memory for gfx10 onwards APUs
>>
>> To decrease VRAM pressure for APUs, put page tables to GTT domain for gfx10 and newer APUs.
>>
>> v2: only enable it for gfx10 and newer APUs (Alex, Christian)
>>
>> Signed-off-by: Yifan Zhang <yifan1.zhang at amd.com<mailto:yifan1.zhang at amd.com>>
>> ---
>>    drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 9 ++++++---
>>    1 file changed, 6 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
>> index 96d601e209b8..4603d87c61a0 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
>> @@ -515,10 +515,13 @@ int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>>           bp.size = amdgpu_vm_pt_size(adev, level);
>>           bp.byte_align = AMDGPU_GPU_PAGE_SIZE;
>>
>> -       if (!adev->gmc.is_app_apu)
>> -               bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
>> -       else
>> +       if (adev->gmc.is_app_apu ||
>> +               ((adev->flags & AMD_IS_APU) &&
>> +               (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 3, 3))))
>>                   bp.domain = AMDGPU_GEM_DOMAIN_GTT;
>> +       else
>> +               bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
>> +
>>
>>           bp.domain = amdgpu_bo_get_preferred_domain(adev, bp.domain);
>>           bp.flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
>> --
>> 2.37.3
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/amd-gfx/attachments/20230829/83e4fac8/attachment-0001.htm>