[RFC PATCH 0/2] add direct IB pool

[Liu, Monk]

Bye the way: if you want to avoid IB test pending by GPU recover, you can move IB test out of TDR routine, so IB test will execute after GPU scheduler resumed with TDR completed .

_____________________________________
Monk Liu|GPU Virtualization Team |AMD
[sig-cloud-gpu]

From: Pan, Xinhui <Xinhui.Pan at amd.com>
Sent: Thursday, March 26, 2020 2:02 PM
To: amd-gfx at lists.freedesktop.org; Liu, Monk <Monk.Liu at amd.com>
Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Kuehling, Felix <Felix.Kuehling at amd.com>; Koenig, Christian <Christian.Koenig at amd.com>
Subject: Re: [RFC PATCH 0/2] add direct IB pool


[AMD Official Use Only - Internal Distribution Only]

yes, IB test and  vram restore will alloc IBs.
I hit this issue for quite a long time ago. We test benchmarks on ARM server which is running android.
Hunders of processes hit too many issues. Panic and memory corruption everywhere.
Now i have a littke time to fix this deadlock.
if you want to repro it, set gpu timeout to 50ms，then run vulkan,ocl, amdgputest, etc together.
I believe you will see more weird issues.
________________________________
From: Liu, Monk <Monk.Liu at amd.com<mailto:Monk.Liu at amd.com>>
Sent: Thursday, March 26, 2020 1:31:04 PM
To: Pan, Xinhui <Xinhui.Pan at amd.com<mailto:Xinhui.Pan at amd.com>>; amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org> <amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>>
Cc: Deucher, Alexander <Alexander.Deucher at amd.com<mailto:Alexander.Deucher at amd.com>>; Kuehling, Felix <Felix.Kuehling at amd.com<mailto:Felix.Kuehling at amd.com>>; Pan, Xinhui <Xinhui.Pan at amd.com<mailto:Xinhui.Pan at amd.com>>; Koenig, Christian <Christian.Koenig at amd.com<mailto:Christian.Koenig at amd.com>>
Subject: RE: [RFC PATCH 0/2] add direct IB pool

That sounds a roughly doable plan to me , although we didn't hit this issue in our virtualization stress test but like a possible issue.

>>> So the ring test above got stuck if no ib to alloc.
Why there is IB alloc happened in ring test ? I remember there is no IB allocated for ring test, are you referring to IB test ?



_____________________________________
Monk Liu|GPU Virtualization Team |AMD


-----Original Message-----
From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org<mailto:amd-gfx-bounces at lists.freedesktop.org>> On Behalf Of xinhui pan
Sent: Thursday, March 26, 2020 10:02 AM
To: amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>
Cc: Deucher, Alexander <Alexander.Deucher at amd.com<mailto:Alexander.Deucher at amd.com>>; Kuehling, Felix <Felix.Kuehling at amd.com<mailto:Felix.Kuehling at amd.com>>; Pan, Xinhui <Xinhui.Pan at amd.com<mailto:Xinhui.Pan at amd.com>>; Koenig, Christian <Christian.Koenig at amd.com<mailto:Christian.Koenig at amd.com>>
Subject: [RFC PATCH 0/2] add direct IB pool

druing gpu recovery, we alloc ibs for ring tests to test if recovery succeed or not.

As gpu recovery parked the gpu scheduler thread, any pending jobs hold the ib resource has no chance to free. So the ring test above got stuck if no ib to alloc.

If we schedule IBs directly in job_submit_direct, we can alloc ibs in the new ib pool. It should have less contention.

If the IB could be freed in time, IOW, not depending on any scheduler, nor any other blocking code. It is better to alloc ibs in direct pool.

xinhui pan (2):
  drm/amdgpu: add direct ib pool
  drm/amdgpu: use new job alloc variation if possible

 drivers/gpu/drm/amd/amdgpu/amdgpu.h         |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c      |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c      | 12 ++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c     |  8 +++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.h     |  3 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c    |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c     |  4 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c     |  6 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c |  3 ++-
 drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c       |  4 ++--
 drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c       |  4 ++--
 13 files changed, 35 insertions(+), 18 deletions(-)

--
2.17.1

_______________________________________________
amd-gfx mailing list
amd-gfx at lists.freedesktop.org<mailto:amd-gfx at lists.freedesktop.org>
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=02%7C01%7Cmonk.liu%40amd.com%7C1f5b1a3ba10a452c9de608d7d129b396%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637207850237679644&sdata=cS7S7a8gDmIgyJNbr4qXSPMZTLwKz0W429Z%2F2Zo6gek%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/amd-gfx/attachments/20200327/487e50a5/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 12243 bytes
Desc: image001.png
URL: <https://lists.freedesktop.org/archives/amd-gfx/attachments/20200327/487e50a5/attachment-0001.png>