[Intel-gfx] [PATCH v2 1/3] drm/i915/guc: keep GuC objects mapped in kernel

[Dave Gordon]

On 18/04/16 12:37, Chris Wilson wrote:
> On Mon, Apr 18, 2016 at 12:28:43PM +0100, Dave Gordon wrote:
>> On 18/04/16 11:25, Chris Wilson wrote:
>>> On Mon, Apr 18, 2016 at 11:15:07AM +0100, Dave Gordon wrote:
>>>> With the new i915_gem_obj_pin_map() interface, it makes sense to keep
>>>> GuC objects (which are always pinned in memory and in the GGTT anyway)
>>>> mapped into kernel address space, rather than mapping and unmapping them
>>>> on each access.
>>>>
>>>> This preliminary patch sets up the pin-and-map for all GuC-specific
>>>> objects, and updates the various setup/shutdown functions to use these
>>>> long-term mappings rather than doing their own kmap_atomic() calls.
>>>>
>>>> v2:
>>>>      Invent & use accessor function i915_object_mapped_address() rather
>>>>      than accessing obj->mapping directly (Tvrtko Ursulin)
>>>>      Use IS_ERR() rather than checking for NULL (Tvrtko Ursulin)
>>>>      Added big comment about struct i915_guc_client & usage of the GEM
>>>>      object that it describes
>>>>
>>>> Cc: <tvrtko.ursulin at intel.com>
>>>> Signed-off-by: Alex Dai <yu.dai at intel.com>
>>>> Signed-off-by: Dave Gordon <david.s.gordon at intel.com>
>>>> ---
>>>>   drivers/gpu/drm/i915/i915_drv.h            | 12 ++++++++++
>>>>   drivers/gpu/drm/i915/i915_guc_submission.c | 38 +++++++++++-------------------
>>>>   drivers/gpu/drm/i915/intel_guc.h           | 24 +++++++++++++++++++
>>>>   3 files changed, 50 insertions(+), 24 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
>>>> index 85102ad..439f149 100644
>>>> --- a/drivers/gpu/drm/i915/i915_drv.h
>>>> +++ b/drivers/gpu/drm/i915/i915_drv.h
>>>> @@ -3025,6 +3025,18 @@ static inline void i915_gem_object_unpin_pages(struct drm_i915_gem_object *obj)
>>>>   void *__must_check i915_gem_object_pin_map(struct drm_i915_gem_object *obj);
>>>>
>>>>   /**
>>>> + * i915_object_mapped_address - address at which a GEM object is mapped
>>>> + * @obj - the object (presumably already mapped into kernel address space)
>>>> + *
>>>> + * Returns the address at which an object has been mapped by a call to
>>>> + * i915_gem_object_pin_map() above, or NULL if not currently mapped and pinned
>>>> + */
>>>> +static inline void *i915_object_mapped_address(struct drm_i915_gem_object *obj)
>>>> +{
>>>> +       return obj->pages_pin_count == 0 ? NULL : obj->mapping;
>>>> +}
>>>
>>> No.
>>
>> The kernel address at which an object is pinned is just as much a
>> property of the object as its GGTT address, so should have a similar
>> function for retrieving it.
>
> An object doesn't have a GGTT, a vma does. The object has vma. To use
> the vma, you must pin it.

Well you obviously have a very different usage model in mind. The GuC 
object lifecycle is:

    guc_obj::create()
    {
       allocate
       get-pages-into-RAM
       pin-pages-in-RAM
       pin-to-gtt
       kmap
    }

    ~guc_obj()
    {
       undo all the above
    }

    guc_obj::do_stuff()
    {
       addr = self.mapped_address()
       ... // use *addr
    }

where all the code relies on the constructor having set up all mappings 
once and for all; whereas it looks like you're thinking:

    do_stuff(gem_obj)
    {
       addr = pin-and-map(gem_obj);
       ... // use *addr
       unpin-and-map(gem_obj)
    }

where each user of a mapping doesn't know about any others that might 
exist, and therefore has to manage pincounting.

This local-knowledge-only model might be more natural if the function 
were called something like i915_gem_object_mapping_create() and returned 
an blob representing a binding between a GEM object and an address space 
-- in other words, a VMA where the "address space" would be "kernel" 
rather than GGTT or PPGTT (and the unmap function would take the 
returned blob, not the GEM object, as the thing-to-be-unmade). Then we 
could have partial mappings, and mappings with different properties 
(e.g. caching) coexisting.

But the current pin-and-map implements *the* mapping as a singleton 
property of a GEM object, so it looks far more natural to set it up once 
and then expect to read its unchanged value at any time thereafter, 
until the mapping is finally destroyed along with the object.

>> The pages_pin_count check is there because once that goes to zero,
>> the object doesn't logically /have/ a memory address, even if
>> 'mapping' is still set.
>>
>> Or was it just the name you objected to?
>
> Name and function. obj->mapping is tied into the shinker which can and
> will run at any time, stealing all locks. Using obj->mapping which
> holding an actual page reference for yourself is broken.

All GuC objects are permanently pinned in RAM and GGTT, and hence immune 
to the shrinker. If the shrinker does touch them, the GuC will break.

>>>> @@ -649,6 +639,9 @@ static void gem_release_guc_obj(struct drm_i915_gem_object *obj)
>>>>   	if (i915_gem_obj_is_pinned(obj))
>>>>   		i915_gem_object_ggtt_unpin(obj);
>>>>
>>>> +	if (i915_object_mapped_address(obj))
>>>> +		i915_gem_object_unpin_map(obj);
>>>
>>> Eh? If you aren't tracking your pin, you can't just randomly free
>>> someone elses.
>>
>> The GuC code exclusively owns this object, so it can't be "someone
>> else's". The pages_pin count here will always be 1 (at present).
>> This was to allow for the possibility that we already chose to
>> release the pin-and-map while leaving the object in the GGTT, in
>> which case the pages_pin_count will be 0. (We don't do this, at
>> present, but see below).
>
> That is bad justification for adding unsafe code.

NOT unsafe, because the GuC code holds the only reference to these 
*private* objects. As I said, no code outside this file will ever access 
these objects. And the code above is the destructor, so it would be a 
really bad idea for anyone else to have a pin-and-mapping of the object 
at this point.

>>>>   	drm_gem_object_unreference(&obj->base);
>>>>   }
>>>>
>>>> @@ -729,6 +722,8 @@ static struct i915_guc_client *guc_client_alloc(struct drm_device *dev,
>>>>   		goto err;
>>>>
>>>>   	client->client_obj = obj;
>>>> +	client->client_base = i915_object_mapped_address(obj);
>>>> +	WARN_ON(!client->client_base);
>>>>   	client->wq_offset = GUC_DB_SIZE;
>>>>   	client->wq_size = GUC_WQ_SIZE;
>>>>
>>>> @@ -841,7 +836,6 @@ static void guc_create_ads(struct intel_guc *guc)
>>>>   	struct guc_policies *policies;
>>>>   	struct guc_mmio_reg_state *reg_state;
>>>>   	struct intel_engine_cs *engine;
>>>> -	struct page *page;
>>>>   	u32 size;
>>>>
>>>>   	/* The ads obj includes the struct itself and buffers passed to GuC */
>>>> @@ -857,9 +851,7 @@ static void guc_create_ads(struct intel_guc *guc)
>>>>
>>>>   		guc->ads_obj = obj;
>>>>   	}
>>>> -
>>>> -	page = i915_gem_object_get_page(obj, 0);
>>>> -	ads = kmap(page);
>>>> +	ads = i915_object_mapped_address(obj);
>>>
>>> You need to explicitly aknowlege that you are using the memory tracked
>>> by the object otherwise it *will* be removed by the shrinker.
>>> So what is wrong with using i915_gem_object_pin_map()?
>>> -Chris
>>
>> The object is pinned-and-mapped just once, when it's created, and
>> kept that way until it's destroyed. Calling pin-and-map here would
>> give it a pin count of two, which would be redundant, and require us
>> to unpin-and-map it again at the end of the function - which just
>> seems a waste of time.
>
> Considering the alternative is an unsound API. Just do it, or pass
> around the mapping in a local you have already pinned.
> -Chris

No, I've decided just not to use the pin-and-map API at all. We really 
only need direct access to the shared doorbell/process descriptor page, 
so we can just kmap the single page instead. This is more economical in 
use of address space, and simpler to keep away from the shrinker.

.Dave.