[PATCH 2/4] drm/gem: adjust per file OOM badness on handling buffers

[Christian König]

From: Christian König <christian.koenig at amd.com>

Large amounts of VRAM are usually not CPU accessible, so they are not mapped
into the processes address space. But since the device drivers usually support
swapping buffers from VRAM to system memory we can still run into an out of
memory situation when userspace starts to allocate to much.

This patch gives the OOM and lower memory killer another hint which process is
holding how many resources.

Signed-off-by: Christian König <christian.koenig at amd.com>
Reviewed-by: Alex Deucher <alexander.deucher at amd.com>
---
 drivers/gpu/drm/drm_gem.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 16a1647..17d14c1 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -282,6 +282,9 @@ drm_gem_handle_delete(struct drm_file *filp, u32 handle)
 
 	if (dev->driver->gem_close_object)
 		dev->driver->gem_close_object(obj, filp);
+
+	atomic_long_sub(obj->size >> PAGE_SHIFT, &filp->filp->f_oom_badness);
+
 	drm_gem_object_handle_unreference_unlocked(obj);
 
 	return 0;
@@ -358,6 +361,9 @@ drm_gem_handle_create_tail(struct drm_file *file_priv,
 		}
 	}
 
+	atomic_long_add(obj->size >> PAGE_SHIFT,
+			&file_priv->filp->f_oom_badness);
+
 	return 0;
 }
 
@@ -717,6 +723,9 @@ drm_gem_object_release_handle(int id, void *ptr, void *data)
 	if (dev->driver->gem_close_object)
 		dev->driver->gem_close_object(obj, file_priv);
 
+	atomic_long_sub(obj->size >> PAGE_SHIFT,
+			&file_priv->filp->f_oom_badness);
+
 	drm_gem_object_handle_unreference_unlocked(obj);
 
 	return 0;
-- 
1.9.1