[Intel-xe] [PATCH v2 1/1] drm/xe/uapi: Use LR abbrev for long-running vms

[Thomas Hellström]

On 11/27/23 19:46, Rodrigo Vivi wrote:
> On Mon, Nov 27, 2023 at 04:03:30PM +0100, Thomas Hellström wrote:
>> Currently we're using "compute mode" for long running VMs using
>> preempt-fences for memory management, and "fault mode" for long
>> running VMs using page faults.
>>
>> Change this to use the terminology "long-running" abbreviated as LR for
>> long-running VMs. These VMs can then either be in preempt-fence mode or
>> fault mode. The user can force fault mode at creation time, but otherwise
>> the driver can choose to use fault- or preempt-fence mode for long-running
>> vms depending on the device capabilities. Initially unless fault-mode is
>> specified, the driver uses preempt-fence mode.
>>
>> v2:
>> - Fix commit message wording and the documentation around
>>    CREATE_FLAG_LR_MODE and CREATE_FLAG_FAULT_MODE
>>
>> Cc: Matthew Brost <matthew.brost at intel.com>
>> Cc: Rodrigo Vivi <rodrigo.vivi at intel.com>
>> Cc: Francois Dugast <francois.dugast at intel.com>
>> Signed-off-by: Thomas Hellström <thomas.hellstrom at linux.intel.com>
>> ---
>>   drivers/gpu/drm/xe/xe_vm.c |  8 ++++----
>>   include/uapi/drm/xe_drm.h  | 23 ++++++++++++++++++++++-
>>   2 files changed, 26 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
>> index aa965e90ce12..6d35067ed8bc 100644
>> --- a/drivers/gpu/drm/xe/xe_vm.c
>> +++ b/drivers/gpu/drm/xe/xe_vm.c
>> @@ -1919,7 +1919,7 @@ static int xe_vm_unbind(struct xe_vm *vm, struct xe_vma *vma,
>>   }
>>   
>>   #define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE | \
>> -				    DRM_XE_VM_CREATE_FLAG_COMPUTE_MODE | \
>> +				    DRM_XE_VM_CREATE_FLAG_LR_MODE | \
>>   				    DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT | \
>>   				    DRM_XE_VM_CREATE_FLAG_FAULT_MODE)
>>   
>> @@ -1955,7 +1955,7 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>>   			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>>   		return -EINVAL;
>>   
>> -	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_COMPUTE_MODE &&
>> +	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
>>   			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>>   		return -EINVAL;
>>   
>> @@ -1972,12 +1972,12 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>>   
>>   	if (args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE)
>>   		flags |= XE_VM_FLAG_SCRATCH_PAGE;
>> -	if (args->flags & DRM_XE_VM_CREATE_FLAG_COMPUTE_MODE)
>> +	if (args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE)
>>   		flags |= XE_VM_FLAG_LR_MODE;
>>   	if (args->flags & DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT)
>>   		flags |= XE_VM_FLAG_ASYNC_DEFAULT;
>>   	if (args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE)
>> -		flags |= XE_VM_FLAG_LR_MODE | XE_VM_FLAG_FAULT_MODE;
>> +		flags |= XE_VM_FLAG_FAULT_MODE;
>>   
>>   	vm = xe_vm_create(xe, flags);
>>   	if (IS_ERR(vm))
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index 88f3aca02b08..2810997a6db4 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -589,8 +589,29 @@ struct drm_xe_vm_create {
>>   	__u64 extensions;
>>   
>>   #define DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE	(1 << 0)
>> -#define DRM_XE_VM_CREATE_FLAG_COMPUTE_MODE	(1 << 1)
>> +	/*
>> +	 * An LR, or Long Running VM accepts exec submissions
>> +	 * to its exec_queues that don't have an upper time limit on
>> +	 * the job execution time. But exec submissions to these
>> +	 * don't allow any of the flags DRM_XE_SYNC_FLAG_SYNCOBJ,
>> +	 * DRM_XE_SYNC_FLAG_TIMELINE_SYNCOBJ, DRM_XE_SYNC_FLAG_DMA_BUF,
>> +	 * used as out-syncobjs, that is, together with DRM_XE_SYNC_FLAG_SIGNAL.
>> +	 * LR VMs can be created in recoverable page-fault mode using
>> +	 * DRM_XE_VM_CREATE_FLAG_FAULT_MODE, if the device supports it.
>> +	 * If that flag is omitted, the UMD can not rely on the slightly
>> +	 * different per-VM overcommit semantics that are enabled by
>> +	 * DRM_XE_VM_CREATE_FLAG_FAULT_MODE (see below), but KMD may
>> +	 * still enable recoverable pagefaults if supported by the device.
>> +	 */
>> +#define DRM_XE_VM_CREATE_FLAG_LR_MODE	        (1 << 1)
>>   #define DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT	(1 << 2)
>> +	/*
>> +	 * DRM_XE_VM_CREATE_FLAG_FAULT_MODE requires also
>> +	 * DRM_XE_VM_CREATE_FLAG_LR_MODE. It allows memory to be allocated
>> +	 * on demand when accessed, and also allows per-VM overcommit of memory.
>> +	 * The xe driver internally uses recoverable pagefaults to implement
>> +	 * this.
> what happens if user selects this flag in a hardware that doesn't support
> recoverable pagefaults?

It will return -EINVAL. Same thing happens if user, for example selects 
scratch page mode.

It also appears like currently the device supports either only pagefault 
mode user VMs or only non-pagefault mode user VMs. Not completely sure why.

/Thomas


>> +	 */
>>   #define DRM_XE_VM_CREATE_FLAG_FAULT_MODE	(1 << 3)
>>   	/** @flags: Flags */
>>   	__u32 flags;
>> -- 
>> 2.41.0
>>