<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <div class="moz-cite-prefix">Am 06.01.21 um 14:02 schrieb Bas
      Nieuwenhuizen:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAP+8YyE2wBHAcn8H-Sr=+kKmE45kUhA=KBJL08ciZJESS4XSmA@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="ltr"><br>
        </div>
        <br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Wed, Jan 6, 2021 at 1:54
            PM Christian König <<a
              href="mailto:christian.koenig@amd.com"
              moz-do-not-send="true">christian.koenig@amd.com</a>>
            wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">Am 06.01.21 um 13:47
            schrieb Joshua Ashton:<br>
            ><br>
            ><br>
            > On 1/6/21 7:52 AM, Christian König wrote:<br>
            >> Am 05.01.21 um 23:31 schrieb Joshua Ashton:<br>
            >>> On 1/5/21 10:10 PM, Alex Deucher wrote:<br>
            >>>> On Tue, Jan 5, 2021 at 5:05 PM Joshua
            Ashton <<a href="mailto:joshua@froggi.es" target="_blank"
              moz-do-not-send="true">joshua@froggi.es</a>> wrote:<br>
            >>>>><br>
            >>>>> Since commit 24562523688b ("Revert
            "drm/amd/amdgpu: set gtt size<br>
            >>>>> according to system memory size
            only""), the GTT size was limited by<br>
            >>>>> 3GiB or VRAM size.<br>
            >>>><br>
            >>>> The commit in question was to fix a hang
            with certain tests on APUs.<br>
            >>>> That should be tested again before we
            re-enable this.  If it is fixed,<br>
            >>>> we should just revert the revert rather
            than special case dGPUs.<br>
            >>>><br>
            >>>> Alex<br>
            >>>><br>
            >>><br>
            >>> I think the commit before the revert
            (ba851eed895c) has some <br>
            >>> fundamental problems:<br>
            >>><br>
            >>> It was always specifying max(3GiB, 3/4ths RAM)
            of GTT, even if that <br>
            >>> wouldn't fit into say, 1GiB or 2GiB of
            available RAM.<br>
            >>><br>
            >>> Limiting GTT to min(max(3GiB, VRAM), 3/4ths
            RAM) size on dGPUs makes <br>
            >>> sense also and is a sensible limit to avoid
            silly situations with <br>
            >>> overallocation and potential OOM.<br>
            >>><br>
            >>> This patch solves both of those issues.<br>
            >><br>
            >> No, Alex is right this approach was already tried
            and it causes <br>
            >> problems.<br>
            >><br>
            >> Additional to that why should this be an issue?
            Even when VRAM is <br>
            >> very small on APUs we still use 3GiB of GTT.<br>
            >><br>
            >> Regards,<br>
            >> Christian.<br>
            ><br>
            > The problem is that 3GiB of GTT isn't enough for most
            modern games.<br>
            <br>
            You seem to misunderstand what the GTT size means here. This
            is the <br>
            amount of memory an application can lock down in a single
            command <br>
            submissions.<br>
            <br>
            It is still possible for the game to use all of system
            memory for <br>
            textures etc... it can just happen that some buffers are
            temporary <br>
            marked as inaccessible for the GPU.<br>
          </blockquote>
          <div><br>
          </div>
          <div>For Vulkan we (both RADV and AMDVLK) use GTT as the total
            size. Usage in modern games is essentially "bindless" so
            there is no way to track at a per-submission level what
            memory needs to be resident. (and even with tracking
            applications are allowed to use all the memory in a single
            draw call, which would be unsplittable anyway ...)<br>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
    Yeah, that is a really good point.<br>
    <br>
    The issue is that we need some limitation since 3/4 of system memory
    is way to much and the max texture size test in piglit can cause a
    system crash.<br>
    <br>
    The alternative is a better OOM handling, so that an application
    which uses to much system memory through the driver stack has a more
    likely chance to get killed. Cause currently that is either X or
    Wayland :(<br>
    <br>
    Christian.<br>
    <br>
    <blockquote type="cite"
cite="mid:CAP+8YyE2wBHAcn8H-Sr=+kKmE45kUhA=KBJL08ciZJESS4XSmA@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_quote">
          <div> <br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <br>
            > My laptop has a 128MiB carveout which is not possible
            to be configured <br>
            > in the BIOS so I am stuck with that size without extra
            kernel <br>
            > parameters which shouldn't be necessary.<br>
            <br>
            Did you ran into problems without the parameter?<br>
            <br>
            ><br>
            > If you dislike the approach of keeping the extra check
            for dGPUs and <br>
            > limiting GTT there, then I would say that we should use<br>
            >     gtt_size = 3/4ths system memory<br>
            > for all devices instead of<br>
            >     gtt_size = max(3/4ths system memory, 3GiB)<br>
            > as it was before the revert, as it is problematic on
            systems with < <br>
            > 3GiB of system memory.<br>
            <br>
            Yeah, that's indeed not a good idea.<br>
            <br>
            Regards,<br>
            Christian.<br>
            <br>
            ><br>
            > - Joshie 🐸✨<br>
            ><br>
            >><br>
            >>><br>
            >>> - Joshie 🐸✨<br>
            >>><br>
            >>>><br>
            >>>>><br>
            >>>>> This is problematic on APUs, especially
            with a small carveout<br>
            >>>>> which can be as low as a fixed 128MiB,
            as there would be very a <br>
            >>>>> limited<br>
            >>>>> 3GiB available for video memory.<br>
            >>>>> This obviously does not meet the
            demands of modern applications.<br>
            >>>>><br>
            >>>>> This patch makes it so the GTT size
            heuristic always uses 3/4ths of<br>
            >>>>> the system memory size on APUs
            (limiting the size by 3GiB/VRAM size<br>
            >>>>> only on devices with dedicated video
            memory).<br>
            >>>>><br>
            >>>>> Fixes: 24562523688b ("Revert
            drm/amd/amdgpu: set gtt size <br>
            >>>>> according to<br>
            >>>>> system memory size only")<br>
            >>>>><br>
            >>>>> Signed-off-by: Joshua Ashton <<a
              href="mailto:joshua@froggi.es" target="_blank"
              moz-do-not-send="true">joshua@froggi.es</a>><br>
            >>>>> ---<br>
            >>>>>  
            drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c |  5 +++--<br>
            >>>>>  
            drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 12 +++++++++---<br>
            >>>>>   2 files changed, 12 insertions(+), 5
            deletions(-)<br>
            >>>>><br>
            >>>>> diff --git
            a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c <br>
            >>>>>
            b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
            >>>>> index 72efd579ec5e..a5a41e9272d6 100644<br>
            >>>>> ---
            a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
            >>>>> +++
            b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c<br>
            >>>>> @@ -192,8 +192,9 @@
            module_param_named(gartsize, amdgpu_gart_size, <br>
            >>>>> uint, 0600);<br>
            >>>>><br>
            >>>>>   /**<br>
            >>>>>    * DOC: gttsize (int)<br>
            >>>>> - * Restrict the size of GTT domain in
            MiB for testing. The <br>
            >>>>> default is -1 (It's VRAM size if 3GB
            < VRAM < 3/4 RAM,<br>
            >>>>> - * otherwise 3/4 RAM size).<br>
            >>>>> + * Restrict the size of GTT domain in
            MiB for testing. The <br>
            >>>>> default is -1 (On APUs this is 3/4th<br>
            >>>>> + * of the system memory; on dGPUs this
            is 3GiB or VRAM sized, <br>
            >>>>> whichever is bigger,<br>
            >>>>> + * with an upper bound of 3/4th of
            system memory.<br>
            >>>>>    */<br>
            >>>>>   MODULE_PARM_DESC(gttsize, "Size of
            the GTT domain in megabytes <br>
            >>>>> (-1 = auto)");<br>
            >>>>>   module_param_named(gttsize,
            amdgpu_gtt_size, int, 0600);<br>
            >>>>> diff --git
            a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c <br>
            >>>>>
            b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
            >>>>> index 4d8f19ab1014..294f26f4f310 100644<br>
            >>>>> ---
            a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
            >>>>> +++
            b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c<br>
            >>>>> @@ -1865,9 +1865,15 @@ int
            amdgpu_ttm_init(struct amdgpu_device <br>
            >>>>> *adev)<br>
            >>>>>                  struct sysinfo si;<br>
            >>>>><br>
            >>>>>                  si_meminfo(&si);<br>
            >>>>> -               gtt_size =
            min(max((AMDGPU_DEFAULT_GTT_SIZE_MB << <br>
            >>>>> 20),<br>
            >>>>> -                             
            adev->gmc.mc_vram_size),<br>
            >>>>> -                             
            ((uint64_t)si.totalram * <br>
            >>>>> si.mem_unit * 3/4));<br>
            >>>>> +               gtt_size =
            (uint64_t)si.totalram * si.mem_unit * 3/4;<br>
            >>>>> +               /* If we have dedicated
            memory, limit our GTT size to<br>
            >>>>> +                * 3GiB or VRAM size,
            whichever is bigger<br>
            >>>>> +                */<br>
            >>>>> +               if (!(adev->flags
            & AMD_IS_APU)) {<br>
            >>>>> +                       gtt_size = <br>
            >>>>> min(max(AMDGPU_DEFAULT_GTT_SIZE_MB
            << 20,<br>
            >>>>> + adev->gmc.mc_vram_size),<br>
            >>>>> +                              
            gtt_size);<br>
            >>>>> +               }<br>
            >>>>>          }<br>
            >>>>>          else<br>
            >>>>>                  gtt_size =
            (uint64_t)amdgpu_gtt_size << 20;<br>
            >>>>> -- <br>
            >>>>> 2.30.0<br>
            >>>>><br>
            >>>>>
            _______________________________________________<br>
            >>>>> amd-gfx mailing list<br>
            >>>>> <a
              href="mailto:amd-gfx@lists.freedesktop.org"
              target="_blank" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a><br>
            >>>>> <a
href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C890f3f3bb9144929d52308d8b2413a35%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637455340521793984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=ix5qMEHXC%2BeOly4OlgZ4mbFPIGz37g0JPawHfh412wE%3D&amp;reserved=0"
              rel="noreferrer" target="_blank" moz-do-not-send="true">https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C890f3f3bb9144929d52308d8b2413a35%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637455340521793984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=ix5qMEHXC%2BeOly4OlgZ4mbFPIGz37g0JPawHfh412wE%3D&amp;reserved=0</a>
            <br>
            >>>>><br>
            >><br>
            <br>
            _______________________________________________<br>
            amd-gfx mailing list<br>
            <a href="mailto:amd-gfx@lists.freedesktop.org"
              target="_blank" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a><br>
            <a
              href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx"
              rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><br>
          </blockquote>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
amd-gfx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>