<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Ah, that issue again. Yeah that was
fixed in the meantime. Please remove the extra
fance_get()/fence_put() for upstreaming.<br>
<br>
If an old branch still has that issue we should backport the
proper fixes and not work around them like this.<br>
<br>
Thanks for looking into it,<br>
Christian.<br>
<br>
Am 11.01.2017 um 09:30 schrieb Liu, Monk:<br>
</div>
<blockquote
cite="mid:BY2PR1201MB11107ADADF57EBE79AB97D7F84660@BY2PR1201MB1110.namprd12.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr"
style="font-size:12pt; color:#000000;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi Christian</p>
<p><br>
</p>
<p>latest updates on the fence grab/release</p>
<p><br>
</p>
<p>I recall the impression of why I added those grab/release
pare:</p>
<p><br>
</p>
<p>I checked the history, the first time I added those pare is
because they are first introduced/implemented in
amd-sriov-4.3, and that branch's fence implement has big gap
between current (4.3 fence doesn't use RCU and kmem_cache)</p>
<p><br>
</p>
<p>in amd-sriov-4.3 branch, without this grab/release around I
found there will be kernel page fault triggered if fence
quickly signaled before fence_wait(), because
amdgpu_fence_process will put the fence and lead to fence
released prior to later's fence_wait().</p>
<p><br>
</p>
<p>but for staging-4.7 code, I didn't found such page fault
after remove those grab/release, and I noticed that the
reason maybe we increases the kref of fence to 2 in
amdgpu_fence_emit ( in 4.3 kernel we only increase it to 1):</p>
<p><br>
</p>
<p>Increasing to 1 is by fence_init, and to 2 is by <span>rcu_assign_pointer(*ptr,fence_get(&fence->base)</span>)</p>
<p><br>
and with this reason the fence is 1 after it signaled.
Agree with you that we should remove those grab/release and
only leave one fence_put() in kiq reg access routines.</p>
<p><br>
</p>
<p>I didn't check the code on detail previously, thanks !</p>
<p><br>
</p>
<p>BR Monk<br>
</p>
<p><br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
color="#000000" face="Calibri, sans-serif"><b>发件人:</b>
amd-gfx <a class="moz-txt-link-rfc2396E" href="mailto:amd-gfx-bounces@lists.freedesktop.org"><amd-gfx-bounces@lists.freedesktop.org></a> 代表
Liu, Monk <a class="moz-txt-link-rfc2396E" href="mailto:Monk.Liu@amd.com"><Monk.Liu@amd.com></a><br>
<b>发送时间:</b> 2017年1月11日 11:06:34<br>
<b>收件人:</b> Christian König; Yu, Xiangliang;
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a><br>
<b>主题:</b> 答复: [V2 04/11] drm/amdgpu/virt: use kiq to access
registers</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">> + fence_get(f);<br>
> + amdgpu_ring_commit(ring);<br>
> + mutex_unlock(&adev->virt.lock);<br>
> +<br>
> + r = fence_wait(f, false);<br>
> + fence_put(f);<br>
<br>
Why do you grab and release an extra fence reference here?<br>
<br>
Christian.<br>
<br>
[ML] e.g. without those grab/release pare, if fence is
signaled right after ring_commit(), then<br>
"amdgpu_fence_process" will be invoked by interrupt and it
will call fence_put() on this fence, so this
<br>
Fence is never valid and all fence_wait on it in sequence
will trigger page fault ...<br>
<br>
BR Monk<br>
<br>
<br>
<br>
-----邮件原件-----<br>
发件人: amd-gfx [<a moz-do-not-send="true"
href="mailto:amd-gfx-bounces@lists.freedesktop.org">mailto:amd-gfx-bounces@lists.freedesktop.org</a>]
代表 Christian König<br>
发送时间: Tuesday, January 10, 2017 9:09 PM<br>
收件人: Yu, Xiangliang; <a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a><br>
主题: Re: [V2 04/11] drm/amdgpu/virt: use kiq to access
registers<br>
<br>
Am 10.01.2017 um 11:00 schrieb Xiangliang Yu:<br>
> For virtualization, it is must for driver to use KIQ to
access <br>
> registers when it is out of GPU full access mode.<br>
><br>
> Signed-off-by: Xiangliang Yu
<a class="moz-txt-link-rfc2396E" href="mailto:Xiangliang.Yu@amd.com"><Xiangliang.Yu@amd.com></a><br>
> ---<br>
> drivers/gpu/drm/amd/amdgpu/Makefile | 2 +-<br>
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 6 +++<br>
> drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 86
++++++++++++++++++++++++++++++<br>
> drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 5 ++<br>
> drivers/gpu/drm/amd/amdgpu/vi.c | 3 ++<br>
> 5 files changed, 101 insertions(+), 1 deletion(-)<br>
> create mode 100644
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c<br>
><br>
> diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile <br>
> b/drivers/gpu/drm/amd/amdgpu/Makefile<br>
> index 4185b03..0b8e470 100644<br>
> --- a/drivers/gpu/drm/amd/amdgpu/Makefile<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/Makefile<br>
> @@ -30,7 +30,7 @@ amdgpu-y += amdgpu_device.o
amdgpu_kms.o \<br>
> atombios_encoders.o amdgpu_sa.o atombios_i2c.o \<br>
> amdgpu_prime.o amdgpu_vm.o amdgpu_ib.o
amdgpu_pll.o \<br>
> amdgpu_ucode.o amdgpu_bo_list.o amdgpu_ctx.o
amdgpu_sync.o \<br>
> - amdgpu_gtt_mgr.o amdgpu_vram_mgr.o<br>
> + amdgpu_gtt_mgr.o amdgpu_vram_mgr.o amdgpu_virt.o<br>
> <br>
> # add asic specific block<br>
> amdgpu-$(CONFIG_DRM_AMDGPU_CIK)+= cik.o cik_ih.o
kv_smc.o kv_dpm.o \ <br>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
<br>
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
> index f82919d..9a2fd3e 100644<br>
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
> @@ -95,6 +95,9 @@ uint32_t amdgpu_mm_rreg(struct
amdgpu_device *adev, uint32_t reg,<br>
> {<br>
> uint32_t ret;<br>
> <br>
> + if (amdgpu_sriov_runtime(adev) &&
!in_interrupt())<br>
> + return amdgpu_virt_kiq_rreg(adev, reg);<br>
> +<br>
> if ((reg * 4) < adev->rmmio_size
&& !always_indirect)<br>
> ret = readl(((void __iomem
*)adev->rmmio) + (reg * 4));<br>
> else {<br>
> @@ -114,6 +117,9 @@ void amdgpu_mm_wreg(struct
amdgpu_device *adev, uint32_t reg, uint32_t v,<br>
> {<br>
> trace_amdgpu_mm_wreg(adev->pdev->device,
reg, v);<br>
> <br>
> + if (amdgpu_sriov_runtime(adev))<br>
> + return amdgpu_virt_kiq_wreg(adev, reg,
v);<br>
> +<br>
> if ((reg * 4) < adev->rmmio_size
&& !always_indirect)<br>
> writel(v, ((void __iomem
*)adev->rmmio) + (reg * 4));<br>
> else {<br>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c <br>
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c<br>
> new file mode 100644<br>
> index 0000000..6520a4e<br>
> --- /dev/null<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c<br>
> @@ -0,0 +1,86 @@<br>
> +/*<br>
> + * Copyright 2017 Advanced Micro Devices, Inc.<br>
> + *<br>
> + * Permission is hereby granted, free of charge, to
any person <br>
> +obtaining a<br>
> + * copy of this software and associated documentation
files (the <br>
> +"Software"),<br>
> + * to deal in the Software without restriction,
including without <br>
> +limitation<br>
> + * the rights to use, copy, modify, merge, publish,
distribute, <br>
> +sublicense,<br>
> + * and/or sell copies of the Software, and to permit
persons to whom <br>
> +the<br>
> + * Software is furnished to do so, subject to the
following conditions:<br>
> + *<br>
> + * The above copyright notice and this permission
notice shall be <br>
> +included in<br>
> + * all copies or substantial portions of the Software.<br>
> + *<br>
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY
OF ANY KIND, <br>
> +EXPRESS OR<br>
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF <br>
> +MERCHANTABILITY,<br>
> + * FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT <br>
> +SHALL<br>
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR
ANY CLAIM, <br>
> +DAMAGES OR<br>
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR <br>
> +OTHERWISE,<br>
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE <br>
> +OR<br>
> + * OTHER DEALINGS IN THE SOFTWARE.<br>
> + */<br>
> +<br>
> +#include "amdgpu.h"<br>
> +#include "amdgpu_virt.h"<br>
> +<br>
> +void amdgpu_virt_init_setting(struct amdgpu_device
*adev) {<br>
> + mutex_init(&adev->virt.lock);<br>
> +}<br>
> +<br>
> +uint32_t amdgpu_virt_kiq_rreg(struct amdgpu_device
*adev, uint32_t <br>
> +reg) {<br>
> + signed long r;<br>
> + uint32_t val;<br>
> + struct fence *f;<br>
> + struct amdgpu_kiq *kiq = &adev->gfx.kiq;<br>
> + struct amdgpu_ring *ring = &kiq->ring;<br>
> +<br>
> + BUG_ON(!ring->funcs->emit_rreg);<br>
> +<br>
> + mutex_lock(&adev->virt.lock);<br>
> + amdgpu_ring_alloc(ring, 32);<br>
> + amdgpu_ring_emit_hdp_flush(ring);<br>
> + amdgpu_ring_emit_rreg(ring, reg);<br>
> + amdgpu_ring_emit_hdp_invalidate(ring);<br>
> + amdgpu_fence_emit(ring, &f);<br>
> + fence_get(f);<br>
> + amdgpu_ring_commit(ring);<br>
> + mutex_unlock(&adev->virt.lock);<br>
> +<br>
> + r = fence_wait(f, false);<br>
> + fence_put(f);<br>
<br>
Why do you grab and release an extra fence reference here?<br>
<br>
Christian.<br>
<br>
> + if (r)<br>
> + DRM_ERROR("wait for kiq fence error:
%ld.\n", r);<br>
> + fence_put(f);<br>
> +<br>
> + val = adev->wb.wb[adev->virt.val_offs];<br>
> +<br>
> + return val;<br>
> +}<br>
> +<br>
> +void amdgpu_virt_kiq_wreg(struct amdgpu_device *adev,
uint32_t reg, <br>
> +uint32_t v) {<br>
> + signed long r;<br>
> + struct fence *f;<br>
> + struct amdgpu_kiq *kiq = &adev->gfx.kiq;<br>
> + struct amdgpu_ring *ring = &kiq->ring;<br>
> +<br>
> + BUG_ON(!ring->funcs->emit_wreg);<br>
> +<br>
> + mutex_lock(&adev->virt.lock);<br>
> + amdgpu_ring_alloc(ring, 32);<br>
> + amdgpu_ring_emit_hdp_flush(ring);<br>
> + amdgpu_ring_emit_wreg(ring, reg, v);<br>
> + amdgpu_ring_emit_hdp_invalidate(ring);<br>
> + amdgpu_fence_emit(ring, &f);<br>
> + fence_get(f);<br>
> + amdgpu_ring_commit(ring);<br>
> + mutex_unlock(&adev->virt.lock);<br>
> +<br>
> + r = fence_wait(f, false);<br>
> + fence_put(f);<br>
> + if (r)<br>
> + DRM_ERROR("wait for kiq fence error:
%ld.\n", r);<br>
> + fence_put(f);<br>
> +}<br>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h <br>
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h<br>
> index 79619b6..24f0590 100644<br>
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h<br>
> @@ -33,6 +33,7 @@<br>
> struct amdgpu_virt {<br>
> uint32_t caps;<br>
> uint32_t val_offs;<br>
> + struct mutex lock;<br>
> };<br>
> <br>
> #define amdgpu_sriov_enabled(adev) \ @@ -59,4 +60,8
@@ static inline <br>
> bool is_virtual_machine(void)<br>
> #endif<br>
> }<br>
> <br>
> +void amdgpu_virt_init_setting(struct amdgpu_device
*adev); uint32_t <br>
> +amdgpu_virt_kiq_rreg(struct amdgpu_device *adev,
uint32_t reg); void <br>
> +amdgpu_virt_kiq_wreg(struct amdgpu_device *adev,
uint32_t reg, <br>
> +uint32_t v);<br>
> +<br>
> #endif<br>
> diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c <br>
> b/drivers/gpu/drm/amd/amdgpu/vi.c index
7350a8f..dc0d4fa 100644<br>
> --- a/drivers/gpu/drm/amd/amdgpu/vi.c<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/vi.c<br>
> @@ -892,6 +892,9 @@ static int
vi_common_early_init(void *handle)<br>
> (amdgpu_ip_block_mask & (1 <<
AMD_IP_BLOCK_TYPE_SMC)))<br>
> smc_enabled = true;<br>
> <br>
> + if (amdgpu_sriov_vf(adev))<br>
> + amdgpu_virt_init_setting(adev);<br>
> +<br>
> adev->rev_id = vi_get_rev_id(adev);<br>
> adev->external_rev_id = 0xFF;<br>
> switch (adev->asic_type) {<br>
<br>
<br>
_______________________________________________<br>
amd-gfx mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a><br>
<a moz-do-not-send="true"
href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><br>
_______________________________________________<br>
amd-gfx mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a><br>
<a moz-do-not-send="true"
href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><br>
</div>
</span></font>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
amd-gfx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/amd-gfx">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>