[PATCH 0/3] reduce system memory requirement for hibernation

[Samuel Zhang]

Modern data center dGPUs are usually equipped with very large VRAM. On
server with such dGPUs(192GB VRAM * 8) and 2TB system memory, hibernate
will fail due to no enough free memory.

The root cause is that during hibernation all VRAM memory get evicted to
GTT or shmem. In both case, it is in system memory and kernel will try to 
copy the pages to hibernation image. In the worst case, this causes 2 
copies of VRAM memory in system memory, 2TB is not enough for the 
hibernation image. 192GB * 8 * 2 = 3TB > 2TB.

The fix includes following 2 changes. With 2 changes, there's much less 
pages needed to be copied to hibernate image and hibernation can succeed.
1. move GTT to shmem after evicting VRAM. then the GTT pages can be freed.
2. force write shmem pages to swap disk and free shmem pages.

After swapout GTT to shmem in hibernation prepare stage, swapin and 
restore BOs in thaw stage takes lots of time(50 mintues observed for 
8 dGPUs). And it's not necessary since the follow-up hibernate stages do 
not use GPU for hibernation successful case. The third patch is just skip 
the BOs restore in thaw stage to reduce the hibernation time.

Samuel Zhang (3):
  drm/amdgpu: move GTT to SHM after eviction for hibernation
  PM: hibernate: shrink shmem pages after dev_pm_ops.prepare()
  drm/amdgpu: skip kfd resume_process for dev_pm_ops.thaw()

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 13 ++++++++++++-
 drivers/gpu/drm/ttm/ttm_resource.c         | 18 ++++++++++++++++++
 include/drm/ttm/ttm_resource.h             |  1 +
 kernel/power/hibernate.c                   | 13 +++++++++++++
 6 files changed, 47 insertions(+), 2 deletions(-)

-- 
2.43.5