<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Am 06.01.21 um 14:02 schrieb Bas
Nieuwenhuizen:<br>
</div>
<blockquote type="cite"
cite="mid:CAP+8YyE2wBHAcn8H-Sr=+kKmE45kUhA=KBJL08ciZJESS4XSmA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jan 6, 2021 at 1:54
PM Christian König <<a
href="mailto:christian.koenig@amd.com"
moz-do-not-send="true">christian.koenig@amd.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Am 06.01.21 um 13:47
schrieb Joshua Ashton:<br>
><br>
><br>
> On 1/6/21 7:52 AM, Christian König wrote:<br>
>> Am 05.01.21 um 23:31 schrieb Joshua Ashton:<br>
>>> On 1/5/21 10:10 PM, Alex Deucher wrote:<br>
>>>> On Tue, Jan 5, 2021 at 5:05 PM Joshua
Ashton <<a href="mailto:joshua@froggi.es" target="_blank"
moz-do-not-send="true">joshua@froggi.es</a>> wrote:<br>
>>>>><br>
>>>>> Since commit 24562523688b ("Revert
"drm/amd/amdgpu: set gtt size<br>
>>>>> according to system memory size
only""), the GTT size was limited by<br>
>>>>> 3GiB or VRAM size.<br>
>>>><br>
>>>> The commit in question was to fix a hang
with certain tests on APUs.<br>
>>>> That should be tested again before we
re-enable this. If it is fixed,<br>
>>>> we should just revert the revert rather
than special case dGPUs.<br>
>>>><br>
>>>> Alex<br>
>>>><br>
>>><br>
>>> I think the commit before the revert
(ba851eed895c) has some <br>
>>> fundamental problems:<br>
>>><br>
>>> It was always specifying max(3GiB, 3/4ths RAM)
of GTT, even if that <br>
>>> wouldn't fit into say, 1GiB or 2GiB of
available RAM.<br>
>>><br>
>>> Limiting GTT to min(max(3GiB, VRAM), 3/4ths
RAM) size on dGPUs makes <br>
>>> sense also and is a sensible limit to avoid
silly situations with <br>
>>> overallocation and potential OOM.<br>
>>><br>
>>> This patch solves both of those issues.<br>
>><br>
>> No, Alex is right this approach was already tried
and it causes <br>
>> problems.<br>
>><br>
>> Additional to that why should this be an issue?
Even when VRAM is <br>
>> very small on APUs we still use 3GiB of GTT.<br>
>><br>
>> Regards,<br>
>> Christian.<br>
><br>
> The problem is that 3GiB of GTT isn't enough for most
modern games.<br>
<br>
You seem to misunderstand what the GTT size means here. This
is the <br>
amount of memory an application can lock down in a single
command <br>
submissions.<br>
<br>
It is still possible for the game to use all of system
memory for <br>
textures etc... it can just happen that some buffers are
temporary <br>
marked as inaccessible for the GPU.<br>
</blockquote>
<div><br>
</div>
<div>For Vulkan we (both RADV and AMDVLK) use GTT as the total
size. Usage in modern games is essentially "bindless" so
there is no way to track at a per-submission level what
memory needs to be resident. (and even with tracking
applications are allowed to use all the memory in a single
draw call, which would be unsplittable anyway ...)<br>
</div>
</div>
</div>
</blockquote>
<br>
Yeah, that is a really good point.<br>
<br>
The issue is that we need some limitation since 3/4 of system memory
is way to much and the max texture size test in piglit can cause a
system crash.<br>
<br>
The alternative is a better OOM handling, so that an application
which uses to much system memory through the driver stack has a more
likely chance to get killed. Cause currently that is either X or
Wayland :(<br>
<br>
Christian.<br>
<br>
<blockquote type="cite"
cite="mid:CAP+8YyE2wBHAcn8H-Sr=+kKmE45kUhA=KBJL08ciZJESS4XSmA@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<br>
> My laptop has a 128MiB carveout which is not possible
to be configured <br>
> in the BIOS so I am stuck with that size without extra
kernel <br>
> parameters which shouldn't be necessary.<br>
<br>
Did you ran into problems without the parameter?<br>
<br>
><br>
> If you dislike the approach of keeping the extra check
for dGPUs and <br>
> limiting GTT there, then I would say that we should use<br>
> gtt_size = 3/4ths system memory<br>
> for all devices instead of<br>
> gtt_size = max(3/4ths system memory, 3GiB)<br>
> as it was before the revert, as it is problematic on
systems with < <br>
> 3GiB of system memory.<br>
<br>
Yeah, that's indeed not a good idea.<br>
<br>
Regards,<br>
Christian.<br>
<br>
><br>
> - Joshie 🐸✨<br>
><br>
>><br>
>>><br>
>>> - Joshie 🐸✨<br>
>>><br>
>>>><br>
>>>>><br>
>>>>> This is problematic on APUs, especially
with a small carveout<br>
>>>>> which can be as low as a fixed 128MiB,
as there would be very a <br>
>>>>> limited<br>
>>>>> 3GiB available for video memory.<br>
>>>>> This obviously does not meet the
demands of modern applications.<br>
>>>>><br>
>>>>> This patch makes it so the GTT size
heuristic always uses 3/4ths of<br>
>>>>> the system memory size on APUs
(limiting the size by 3GiB/VRAM size<br>
>>>>> only on devices with dedicated video
memory).<br>
>>>>><br>
>>>>> Fixes: 24562523688b ("Revert
drm/amd/amdgpu: set gtt size <br>
>>>>> according to<br>
>>>>> system memory size only")<br>
>>>>><br>
>>>>> Signed-off-by: Joshua Ashton <<a
href="mailto:joshua@froggi.es" target="_blank"
moz-do-not-send="true">joshua@froggi.es</a>><br>
>>>>> ---<br>
>>>>>
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 5 +++--<br>
>>>>>
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 12 +++++++++---<br>
>>>>> 2 files changed, 12 insertions(+), 5
deletions(-)<br>
>>>>><br>
>>>>> diff --git
a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c <br>
>>>>>
b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
>>>>> index 72efd579ec5e..a5a41e9272d6 100644<br>
>>>>> ---
a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
>>>>> +++
b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
>>>>> @@ -192,8 +192,9 @@
module_param_named(gartsize, amdgpu_gart_size, <br>
>>>>> uint, 0600);<br>
>>>>><br>
>>>>> /**<br>
>>>>> * DOC: gttsize (int)<br>
>>>>> - * Restrict the size of GTT domain in
MiB for testing. The <br>
>>>>> default is -1 (It's VRAM size if 3GB
< VRAM < 3/4 RAM,<br>
>>>>> - * otherwise 3/4 RAM size).<br>
>>>>> + * Restrict the size of GTT domain in
MiB for testing. The <br>
>>>>> default is -1 (On APUs this is 3/4th<br>
>>>>> + * of the system memory; on dGPUs this
is 3GiB or VRAM sized, <br>
>>>>> whichever is bigger,<br>
>>>>> + * with an upper bound of 3/4th of
system memory.<br>
>>>>> */<br>
>>>>> MODULE_PARM_DESC(gttsize, "Size of
the GTT domain in megabytes <br>
>>>>> (-1 = auto)");<br>
>>>>> module_param_named(gttsize,
amdgpu_gtt_size, int, 0600);<br>
>>>>> diff --git
a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c <br>
>>>>>
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
>>>>> index 4d8f19ab1014..294f26f4f310 100644<br>
>>>>> ---
a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
>>>>> +++
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
>>>>> @@ -1865,9 +1865,15 @@ int
amdgpu_ttm_init(struct amdgpu_device <br>
>>>>> *adev)<br>
>>>>> struct sysinfo si;<br>
>>>>><br>
>>>>> si_meminfo(&si);<br>
>>>>> - gtt_size =
min(max((AMDGPU_DEFAULT_GTT_SIZE_MB << <br>
>>>>> 20),<br>
>>>>> -
adev->gmc.mc_vram_size),<br>
>>>>> -
((uint64_t)si.totalram * <br>
>>>>> si.mem_unit * 3/4));<br>
>>>>> + gtt_size =
(uint64_t)si.totalram * si.mem_unit * 3/4;<br>
>>>>> + /* If we have dedicated
memory, limit our GTT size to<br>
>>>>> + * 3GiB or VRAM size,
whichever is bigger<br>
>>>>> + */<br>
>>>>> + if (!(adev->flags
& AMD_IS_APU)) {<br>
>>>>> + gtt_size = <br>
>>>>> min(max(AMDGPU_DEFAULT_GTT_SIZE_MB
<< 20,<br>
>>>>> + adev->gmc.mc_vram_size),<br>
>>>>> +
gtt_size);<br>
>>>>> + }<br>
>>>>> }<br>
>>>>> else<br>
>>>>> gtt_size =
(uint64_t)amdgpu_gtt_size << 20;<br>
>>>>> -- <br>
>>>>> 2.30.0<br>
>>>>><br>
>>>>>
_______________________________________________<br>
>>>>> amd-gfx mailing list<br>
>>>>> <a
href="mailto:amd-gfx@lists.freedesktop.org"
target="_blank" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a><br>
>>>>> <a
href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Cchristian.koenig%40amd.com%7C890f3f3bb9144929d52308d8b2413a35%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637455340521793984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ix5qMEHXC%2BeOly4OlgZ4mbFPIGz37g0JPawHfh412wE%3D&reserved=0"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Cchristian.koenig%40amd.com%7C890f3f3bb9144929d52308d8b2413a35%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637455340521793984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ix5qMEHXC%2BeOly4OlgZ4mbFPIGz37g0JPawHfh412wE%3D&reserved=0</a>
<br>
>>>>><br>
>><br>
<br>
_______________________________________________<br>
amd-gfx mailing list<br>
<a href="mailto:amd-gfx@lists.freedesktop.org"
target="_blank" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a><br>
<a
href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><br>
</blockquote>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
amd-gfx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a>
</pre>
</blockquote>
<br>
</body>
</html>