[PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails

[Daniel Vetter]

On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote:
> GPU buffers can be quite large, so userspace is expected to deal with
> allocation failure. Don't trigger the OOM killer when page allocation for
> the GEM objects fails, as this opens an easy possiblity for unprivileged
> applications to DOS the system,a s the shmem pages are not fully accounted
> to the allocating process.
> 
> Signed-off-by: Lucas Stach <dev at lynxeye.de>
> ---
>  drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
> index f0efc5d..4c53508 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
> @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
>  		 * going to pin these pages.
>  		 */
>  		mapping = obj->filp->f_mapping;
> -		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
> +		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
> +				     __GFP_NORETRY | __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done
this patch in 4.12 and totally regret it, because now gpu tasks run out of
memory with plenty of (gpu) memory available that could be reaped.

There's some discussions going on with Michal Hocko and Chris Wilson
about possible solutions for this without so much hacks.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch