<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 2024-04-23 18:17, Felix Kuehling
wrote:<br>
</div>
<blockquote type="cite" cite="mid:00003b2f-5f31-4bf3-828a-8d7eea792563@amd.com">
<br>
On 2024-04-23 11:28, Philip Yang wrote:
<br>
<blockquote type="cite">RDMA device with limited scatter-gather
ability requires contiguous VRAM
<br>
buffer allocation for RDMA peer direct support.
<br>
<br>
Add a new KFD alloc memory flag and store as bo alloc flag
<br>
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS. When pin this bo to export
for RDMA
<br>
peerdirect access, this will set TTM_PL_FLAG_CONTIFUOUS flag,
and ask
<br>
VRAM buddy allocator to get contiguous VRAM.
<br>
<br>
Signed-off-by: Philip Yang <a class="moz-txt-link-rfc2396E" href="mailto:Philip.Yang@amd.com"><Philip.Yang@amd.com></a>
<br>
---
<br>
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 ++++
<br>
include/uapi/linux/kfd_ioctl.h | 1 +
<br>
2 files changed, 5 insertions(+)
<br>
<br>
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
<br>
index 0ae9fd844623..ef9154043757 100644
<br>
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
<br>
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
<br>
@@ -1712,6 +1712,10 @@ int
amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
<br>
alloc_flags =
AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE;
<br>
alloc_flags |= (flags &
KFD_IOC_ALLOC_MEM_FLAGS_PUBLIC) ?
<br>
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED : 0;
<br>
+
<br>
+ /* For contiguous VRAM allocation */
<br>
+ if (flags &
KFD_IOC_ALLOC_MEM_FLAGS_CONTIGUOUS_BEST_EFFORT)
<br>
+ alloc_flags |=
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
<br>
}
<br>
xcp_id = fpriv->xcp_id == AMDGPU_XCP_NO_PARTITION ?
<br>
0 : fpriv->xcp_id;
<br>
diff --git a/include/uapi/linux/kfd_ioctl.h
b/include/uapi/linux/kfd_ioctl.h
<br>
index 2040a470ddb4..c1394c162d4e 100644
<br>
--- a/include/uapi/linux/kfd_ioctl.h
<br>
+++ b/include/uapi/linux/kfd_ioctl.h
<br>
@@ -407,6 +407,7 @@ struct kfd_ioctl_acquire_vm_args {
<br>
#define KFD_IOC_ALLOC_MEM_FLAGS_COHERENT (1 << 26)
<br>
#define KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED (1 << 25)
<br>
#define KFD_IOC_ALLOC_MEM_FLAGS_EXT_COHERENT (1 <<
24)
<br>
+#define KFD_IOC_ALLOC_MEM_FLAGS_CONTIGUOUS_BEST_EFFORT (1
<< 23)
<br>
</blockquote>
<br>
If I understand it correctly, AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS
was redefined to mean "best effort". Maybe we can drop the
explicit "BEST_EFFORT" from this flag as well to keep the name to
a reasonable length.
<br>
</blockquote>
<p>yes, AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS is redefined, to implement
"best effort" without adding new upstream GEM flag, so we may get
scattered allocation if contiguous allocation failed. If we drop
the "BEST_EFFORT" from flag name, this may mislead the users.</p>
<p>Regards,</p>
<p>Philip </p>
<blockquote type="cite" cite="mid:00003b2f-5f31-4bf3-828a-8d7eea792563@amd.com">
<br>
Regards,
<br>
Felix
<br>
<br>
<br>
<blockquote type="cite"> /* Allocate memory for later SVM
(shared virtual memory) mapping.
<br>
*
<br>
</blockquote>
</blockquote>
</body>
</html>