[Intel-gfx] [RFC] Cross-driver ww transaction lock lists

[Thomas Hellström (Intel)]

Hi,

On 4/13/21 9:57 AM, Christian König wrote:
>
>
> Am 13.04.21 um 09:50 schrieb Thomas Hellström:
>> Hi!
>>
>> During the dma_resv conversion of the i915 driver, we've been using ww
>> transaction lock lists to keep track of ww_mutexes that are locked
>> during the transaction so that they can be batch unlocked at suitable
>> locations. Including also the LMEM/VRAM eviction we've ended up with
>> two static lists per transaction context; one typically unlocked at the
>> end of transaction and one initialized before and unlocked after each
>> buffer object validate. This enables us to do sleeping locking at
>> eviction and keep objects locked on the eviction list until we
>> eventually succeed allocating memory (modulo minor flaws of course).
>>
>> It would be beneficial with the i915 TTM conversion to be able to
>> introduce a similar functionality that would work in ttm but also
>> cross-driver in, for example move_notify. It would also be beneficial
>> to be able to put any dma_resv ww mutex on the lists, and not require
>> it to be embedded in a particular object type.
>>
>> I started scetching on some utilities for this. For TTM, for example,
>> the idea would be to pass a list head for the ww transaction lock list
>> in the ttm_operation_ctx. A function taking a ww_mutex could then
>> either attach a grabbed lock to the list for batch unlocking, or be
>> responsible for unlocking when the lock's scope is exited. It's also
>> possible to create sublists if so desired. I believe the below would be
>> sufficient to cover the i915 functionality.
>>
>> Any comments and suggestions appreciated!
>
> ah yes Daniel and I haven been discussing something like this for years.
>
> I also came up with rough implementation, but we always ran into 
> lifetime issues.
>
> In other words the ww_mutexes which are on the list would need to be 
> kept alive until unlocked.

Yes, the idea here is that the locker takes whatever reference needed to 
put the lock on the list and keep it alive, and if that's not possible 
for some reason, don't put the object on the list, but in the latter 
case we need to solve the contended case. If needed I have a ww_mutex 
api addition that does that...

>
> Because of this we kind of backed up and said we would need this on 
> the GEM level instead of working with dma_resv objects.

And that's what the ops are for. The implementation doesn't care about 
the underlying object. For TTM, we'd probably want to take a refcount on 
the embedded GEM object.

/Thomas


>
> Regards,
> Christian.
>
>>
>> 8<------------------------------------------------------
>>
>> #ifndef _TRANSACTION_LOCKLIST_H_
>> #define _TRANSACTION_LOCKLIST_H_
>>
>> struct trans_lockitem;
>>
>> /**
>>   * struct trans_locklist_ops - Ops structure for the ww locklist
>> functionality.
>>   *
>>   * Typically a const struct trans_locklist_ops is defined for each type
>> that
>>   * embeds a struct trans_lockitem, or hav a struct trans_lockitem
>> pointing
>>   * at it using the private pointer. It can be a buffer object,
>> reservation
>>   * object, a single ww_mutex or even a sublist of trans_lockitems.
>>   */
>> struct trans_locklist_ops {
>>     /**
>>      * slow_lock: Slow lock to relax the transaction. Only used by
>>      * a contending lock item.
>>      * @item: The struct trans_lockitem to lock
>>      * @intr: Whether to lock interruptible
>>      *
>>      * Return: -ERESTARTSYS: Hit a signal when relaxing,
>>      * -EAGAIN, relaxing succesful, but the contending lock
>>      * remains unlocked.
>>      */
>>     int (*slow_lock) (struct trans_lockitem *item, bool intr);
>>
>>     /**
>>      * unlock: Unlock.
>>      * @item: The struct trans_lockitem to unlock.
>>      */
>>     void (*unlock) (struct trans_lockitem *item);
>>
>>     /**
>>      * unlock: Unlock.
>>      * @item: The struct trans_lockitem to put. This function may
>> be NULL.
>>      */
>>     void (*put) (struct trans_lockitem *item);
>> };
>>
>> /**
>>   * struct trans_lockitem
>>   * @ops: Pointer to an Ops structure for this lockitem.
>>   * @link: List link for the transaction locklist.
>>   * @private: Private info.
>>   * @relax_unlock: Unlock contending lock after relaxation since it is
>>   * likely not needed after a transaction restart.
>>   *
>>   * A struct trans_lockitem typically represents a single lock, but is
>>   * generic enough to represent a sublist of locks. It can either be
>>   * embedded, or allocated on demand. A kmem_cache might be beneficial.
>>   */
>> struct trans_lockitem {
>>     const struct trans_locklist_ops *ops;
>>     struct list_head link;
>>     u32 relax_unlock:1;
>>     void *private;
>> };
>>
>> /* unlock example */
>> static inline void trans_unlock_put_all(struct list_head *list)
>> {
>>     struct trans_lockitem *lock, *next;
>>
>>     list_for_each_entry_safe(lock, next, typeof(*lock), link) {
>>         lock->ops->unlock(lock);
>>         list_del_init(&lock->link);
>>         if (lock->ops_put)
>>             lock->ops->put(lock);
>>     }
>> }
>>
>> /* Backoff example */
>> static inline int __must_check trans_backoff(struct list_head *list,
>> bool intr,
>>                          struct trans_lockitem
>> *contending)
>> {
>>     int ret = 0;
>>
>>     trans_unlock_put_all(list);
>>     if (contending) {
>>         ret = contending->ops->slow_lock(contending, intr);
>>         if (!ret && contending->relax_unlock)
>>             contending->unlock(contending);
>>         if (ret == -EAGAIN)
>>             ret = 0;
>>         contending->ops->put(contending);
>>     }
>>
>>     return ret;
>> }
>>
>>
>> #endif _TRANSACTION_LOCKLIST_
>>
>>
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx