[Intel-gfx] [PATCH] Fix system lockup when a client tries to allocate a huge BO

[Simon Farnsworth]

From: Simon Farnsworth <simon.farnsworth at onelan.co.uk>

Fixes bugs.fdo bug #22601

UXA tries to back all pixmaps with BOs; we have some code that tries
to allocate insanely large pixmaps (30,000 by 30,000 pixels). The
resulting pixmap is 3,600,000,000 bytes large, which is obviously too
big for a BO on 32-bit.

There were two parts to the bug; firstly, logbase2 came back with the
wrong answer for values that were larger than INT_MAX but less than
ULONG_MAX.

Secondly, drm_intel_gem_bo_alloc_internal didn't validate the results
it got from logbase2. Fix this by looking for cases where the result
was obviously invalid, and failing fast.
---
 libdrm/intel/intel_bufmgr_gem.c |   15 ++++++++++++---
 1 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/libdrm/intel/intel_bufmgr_gem.c b/libdrm/intel/intel_bufmgr_gem.c
index c25fc4c..556bcf2 100644
--- a/libdrm/intel/intel_bufmgr_gem.c
+++ b/libdrm/intel/intel_bufmgr_gem.c
@@ -204,16 +204,19 @@ static void
 drm_intel_gem_bo_unreference(drm_intel_bo *bo);
 
 static int
-logbase2(int n)
+logbase2(unsigned long n)
 {
-   int i = 1;
+   unsigned long i = 1;
    int log2 = 0;
 
-   while (n > i) {
+   while (n > i && i != 0) {
       i *= 2;
       log2++;
    }
 
+   if (i == 0)
+      log2++;
+
    return log2;
 }
 
@@ -348,6 +351,12 @@ drm_intel_gem_bo_alloc_internal(drm_intel_bufmgr *bufmgr, const char *name,
     bo_size = 1 << logbase2(size);
     if (bo_size < page_size)
 	bo_size = page_size;
+
+    /* If we're up near the overflow point of an unsigned long, give
+     * up now - there is no way we can allocate this much memory. */
+    if (bo_size < size)
+       return NULL;
+
     bucket = drm_intel_gem_bo_bucket_for_size(bufmgr_gem, bo_size);
 
     /* If we don't have caching at this size, don't actually round the
-- 
1.5.4.1