[Mesa-dev] [PATCH] anv: Limit VkDeviceMemory objects to 2GB

[Chris Wilson]

On Tue, Apr 11, 2017 at 10:58:41AM -0700, Jason Ekstrand wrote:
> Cc: "Juan A. Suárez" <jasuarez at igalia.com>
> ---
>  src/intel/vulkan/anv_device.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/src/intel/vulkan/anv_device.c b/src/intel/vulkan/anv_device.c
> index 35ef4c4..b24c739 100644
> --- a/src/intel/vulkan/anv_device.c
> +++ b/src/intel/vulkan/anv_device.c
> @@ -1539,6 +1539,23 @@ VkResult anv_AllocateMemory(
>     assert(pAllocateInfo->memoryTypeIndex == 0 ||
>            (!device->info.has_llc && pAllocateInfo->memoryTypeIndex < 2));
>  
> +   /* The kernel relocation API has a limitation of a 32-bit delta value
> +    * applied to the address before it is written which, in spite of it being
> +    * unsigned, is treated as signed .  Because of the way that this maps to
> +    * the Vulkan API, we cannot handle an offset into a buffer that does not
> +    * fit into a signed 31 bits.

Correct. Other users take advantage of the negative deltas.

> The only mechanism we have for dealing with
> +    * this at the moment is to limit all VkDeviceMemory objects to a maximum
> +    * of 2GB each.  The Vulkan spec allows us to do this:
> +    *
> +    *    "Some platforms may have a limit on the maximum size of a single
> +    *    allocation. For example, certain systems may fail to create
> +    *    allocations with a size greater than or equal to 4GB. Such a limit is
> +    *    implementation-dependent, and if such a failure occurs then the error
> +    *    VK_ERROR_OUT_OF_DEVICE_MEMORY should be returned."
> +    */
> +   if (pAllocationInfo->allocationSize > (1ull << 31))
> +      return VK_ERROR_OUT_OF_HOST_MEMORY;

Hmm. So how do we lift this? Is this a complete blocker to large object
support, or is this only the slab allocator that is limited? The
phrasing (all VkDeviceMemory) suggests that this is imposing an absolute
limit to any object size.

Otoh, there are good reasons why you probably want to stitch together
smaller objects to emulate a huge object (e.g. sparse).

If this is a blocker, there is enough space inside
drm_i915_gem_relocation_entry to support a s65 (full u64 delta plus
direction bit) by sacrificing read/write domain and reusing delta as a
flag (with an exec flag to indicate the change in meaning). Or you may
just declare this the final straw and all large objects will be
softpinned and relocation-less.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre