<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
[putting Harry on BCC, sorry for the noise]<br>
<br>
Yeah, that is clearly a bug in the KFD.<br>
<br>
During the second eviction the hw should already be disabled, so we
don't have any SDMA or similar to evict BOs any more and can only
copy them with the CPU.<br>
<br>
@Felix what workqueue do you guys use for the restore work? I've
just double checked and on the system workqueues you explicitly need
to specify that stuff is freezable. E.g. use system_freezable_wq
instead of system_wq.<br>
<br>
Alternatively as Xinhui mentioned it might be necessary to flush all
restore work before the first eviction phase or we have the chance
that BOs are moved back into VRAM again.<br>
<br>
Regards,<br>
Christian.<br>
<br>
<div class="moz-cite-prefix">Am 14.09.23 um 03:54 schrieb Pan,
Xinhui:<br>
</div>
<blockquote type="cite"
cite="mid:DM4PR12MB5165B9687FBB14DAEF11944187F7A@DM4PR12MB5165.namprd12.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:"Microsoft YaHei";
panose-1:2 11 5 3 2 2 4 2 2 4;}@font-face
{font-family:"\@Microsoft YaHei";}@font-face
{font-family:Aptos;}@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<p
style="font-family:Arial;font-size:10pt;color:#0000FF;margin:5pt;font-style:normal;font-weight:normal;text-decoration:none;"
align="Left">
[AMD Official Use Only - General]<br>
</p>
<br>
<div>
<div class="WordSection1">
<p class="MsoNormal">I just make one debug patch to show busy
BO’s alloc-trace when the eviction fails in suspend.<o:p></o:p></p>
<p class="MsoNormal">And dmesg log attached.<o:p></o:p></p>
<p class="MsoNormal">Looks like they are just kfd user Bos and
locked by evict/restore work.<o:p></o:p></p>
<p class="MsoNormal">So in kfd suspend callback, it really
need to flush the evict/restore work before HW fini as it do
now.<o:p></o:p></p>
<p class="MsoNormal">That is why the first very early eviction
fails and the second eviction succeed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal">xinhui<o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Pan, Xinhui <br>
<b>Sent:</b> Thursday, September 14, 2023 8:02 AM<br>
<b>To:</b> Koenig, Christian
<a class="moz-txt-link-rfc2396E" href="mailto:Christian.Koenig@amd.com"><Christian.Koenig@amd.com></a>; Kuehling, Felix
<a class="moz-txt-link-rfc2396E" href="mailto:Felix.Kuehling@amd.com"><Felix.Kuehling@amd.com></a>; Christian König
<a class="moz-txt-link-rfc2396E" href="mailto:ckoenig.leichtzumerken@gmail.com"><ckoenig.leichtzumerken@gmail.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>; Wentland, Harry
<a class="moz-txt-link-rfc2396E" href="mailto:Harry.Wentland@amd.com"><Harry.Wentland@amd.com></a><br>
<b>Cc:</b> Deucher, Alexander
<a class="moz-txt-link-rfc2396E" href="mailto:Alexander.Deucher@amd.com"><Alexander.Deucher@amd.com></a>; Fan, Shikang
<a class="moz-txt-link-rfc2396E" href="mailto:Shikang.Fan@amd.com"><Shikang.Fan@amd.com></a><br>
<b>Subject:</b> RE: <span
style="font-family:"Microsoft
YaHei",sans-serif" lang="ZH-CN">
回复</span>: [PATCH] drm/amdgpu: Ignore first evction
failure during suspend<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Chris,<o:p></o:p></p>
<p class="MsoNormal">I can dump these busy BOs with their
alloc/free stack later today.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">BTW, the two evictions and the kfd
suspend are all called before hw_fini. IOW, between phase 1
and phase 2. SDMA is turned only in phase2. So current code
works fine maybe.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Koenig, Christian <<a
href="mailto:Christian.Koenig@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Christian.Koenig@amd.com</a>>
<br>
<b>Sent:</b> Wednesday, September 13, 2023 10:29 PM<br>
<b>To:</b> Kuehling, Felix <<a
href="mailto:Felix.Kuehling@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Felix.Kuehling@amd.com</a>>;
Christian König <<a
href="mailto:ckoenig.leichtzumerken@gmail.com"
moz-do-not-send="true" class="moz-txt-link-freetext">ckoenig.leichtzumerken@gmail.com</a>>;
Pan, Xinhui <<a href="mailto:Xinhui.Pan@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Xinhui.Pan@amd.com</a>>;
<a href="mailto:amd-gfx@lists.freedesktop.org"
moz-do-not-send="true" class="moz-txt-link-freetext">amd-gfx@lists.freedesktop.org</a>;
Wentland, Harry <<a
href="mailto:Harry.Wentland@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Harry.Wentland@amd.com</a>><br>
<b>Cc:</b> Deucher, Alexander <<a
href="mailto:Alexander.Deucher@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Alexander.Deucher@amd.com</a>>;
Fan, Shikang <<a href="mailto:Shikang.Fan@amd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">Shikang.Fan@amd.com</a>><br>
<b>Subject:</b> Re: <span
style="font-family:"Microsoft
YaHei",sans-serif" lang="ZH-CN">
回复</span>: [PATCH] drm/amdgpu: Ignore first evction
failure during suspend<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">[+Harry]<o:p></o:p></p>
<div>
<p class="MsoNormal">Am 13.09.23 um 15:54 schrieb Felix
Kuehling:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On 2023-09-13 4:07, Christian König
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">[+Fleix]<br>
<br>
Well that looks like quite a serious bug.<br>
<br>
If I'm not completely mistaken the KFD work item tries
to restore the process by moving BOs into memory even
after the suspend freeze. Normally work items are frozen
together with the user space processes unless explicitly
marked as not freezable.<br>
<br>
That this causes problem during the first eviction phase
is just the tip of the iceberg here. If a BO is moved
into invisible memory during this we wouldn't be able to
get it out of that in the second phase because SDMA and
hw is already turned off.<br>
<br>
@Felix any idea how that can happen? Have you guys
marked a work item / work queue as not freezable?<o:p></o:p></p>
</blockquote>
<p>We don't set anything to non-freezable in KFD.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>Regards,<br>
Felix<o:p></o:p></p>
<p><o:p> </o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Or maybe the display guys?<o:p></o:p></p>
</blockquote>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Do you guys in the display do any delayed update in a work
item which is marked as not-freezable?<br>
<br>
Otherwise I have absolutely no idea what's going on here.<br>
<br>
Thanks,<br>
Christian.<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
@Xinhui please investigate what work item that is and
where that is coming from. Something like "if
(adev->in_suspend) dump_stack();" in the right place
should probably do it.<br>
<br>
Thanks,<br>
Christian.<o:p></o:p></p>
<div>
<p class="MsoNormal">Am 13.09.23 um 07:13 schrieb Pan,
Xinhui:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p style="margin:5.0pt"><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:blue">[AMD
Official Use Only - General]<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">I
notice that only user space process are frozen
on my side. kthread and workqueue keeps
running. Maybe some kernel configs are not
enabled.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">I
made one module which just prints something like
i++ with mutex lock both in workqueue and
kthread. I paste some logs below.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438619.696196]
XH: 14 from workqueue
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438619.700193]
XH: 15 from kthread<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.394335]
PM: suspend entry (deep)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.399619]
Filesystems sync: 0.001 seconds<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.403887]
PM: Preparing system for sleep (deep)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.409299]
Freezing user space processes<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.414862]
Freezing user space processes completed
(elapsed 0.001 seconds)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.421881]
OOM killer disabled.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.425197]
Freezing remaining freezable tasks<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.430890]
Freezing remaining freezable tasks completed
(elapsed 0.001 seconds)<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438620.438348]
PM: Suspending system (deep)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">.....<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438623.746038]
PM: suspend of devices complete after 3303.137
msecs
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438623.752125]
PM: start suspend of devices complete after
3309.713 msecs<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438623.758722]
PM: suspend debug: Waiting for 5 second(s).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438623.792166]
XH: 22 from kthread<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">[438623.824140]
XH: 23 from workqueue<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">So
BOs definitely can be in use during suspend.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Even
if kthread or workqueue can be stopped with one
special kernel config. I think suspend can only
stop the workqueue with its callback finish.
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">otherwise
something like below makes things crazy.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">LOCK
BO<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">do
something<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">
-> schedule or wait, anycode might sleep.
Stopped by suspend now? no, i think.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">UNLOCK
BO<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">I
do tests with cmds below.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">echo
devices > /sys/power/pm_test
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">echo
0 > /sys/power/pm_async<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">echo
1 > /sys/power/pm_print_times<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">echo
1 > /sys/power/pm_debug_messages<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">echo
1 >
/sys/module/amdgpu/parameters/debug_evictions<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">./kfd.sh
--gtest_filter=KFDEvictTest.BasicTest<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">pm-suspend<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">thanks<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">xinhui<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr width="98%" size="1" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span
style="font-family:"Microsoft
YaHei",sans-serif;color:black"
lang="ZH-CN">发件人</span><span
style="color:black">:</span></b><span
style="color:black"> Christian König
<a
href="mailto:ckoenig.leichtzumerken@gmail.com"
moz-do-not-send="true"><ckoenig.leichtzumerken@gmail.com></a><br>
</span><b><span style="font-family:"Microsoft
YaHei",sans-serif;color:black"
lang="ZH-CN">发送时间</span><span
style="color:black">:</span></b><span
style="color:black"> 2023</span><span
style="font-family:"Microsoft
YaHei",sans-serif;color:black" lang="ZH-CN">年</span><span
style="color:black">9</span><span
style="font-family:"Microsoft
YaHei",sans-serif;color:black" lang="ZH-CN">月</span><span
style="color:black">12</span><span
style="font-family:"Microsoft
YaHei",sans-serif;color:black" lang="ZH-CN">日</span><span
style="color:black"> 17:01<br>
</span><b><span style="font-family:"Microsoft
YaHei",sans-serif;color:black"
lang="ZH-CN">收件人</span><span
style="color:black">:</span></b><span
style="color:black"> Pan, Xinhui
<a href="mailto:Xinhui.Pan@amd.com"
moz-do-not-send="true"><Xinhui.Pan@amd.com></a>;
<a href="mailto:amd-gfx@lists.freedesktop.org"
moz-do-not-send="true"
class="moz-txt-link-freetext">
amd-gfx@lists.freedesktop.org</a> <a
href="mailto:amd-gfx@lists.freedesktop.org"
moz-do-not-send="true">
<amd-gfx@lists.freedesktop.org></a><br>
</span><b><span style="font-family:"Microsoft
YaHei",sans-serif;color:black"
lang="ZH-CN">抄送</span><span
style="color:black">:</span></b><span
style="color:black"> Deucher, Alexander
<a href="mailto:Alexander.Deucher@amd.com"
moz-do-not-send="true"><Alexander.Deucher@amd.com></a>;
Koenig, Christian
<a href="mailto:Christian.Koenig@amd.com"
moz-do-not-send="true"><Christian.Koenig@amd.com></a>;
Fan, Shikang
<a href="mailto:Shikang.Fan@amd.com"
moz-do-not-send="true"><Shikang.Fan@amd.com></a><br>
</span><b><span style="font-family:"Microsoft
YaHei",sans-serif;color:black"
lang="ZH-CN">主题</span><span
style="color:black">:</span></b><span
style="color:black"> Re: [PATCH] drm/amdgpu:
Ignore first evction failure during suspend</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">When
amdgpu_device_suspend() is called processes
should be frozen
<br>
already. In other words KFD queues etc... should
already be idle.<br>
<br>
So when the eviction fails here we missed
something previously and that <br>
in turn can cause tons amount of problems.<br>
<br>
So ignoring those errors is most likely not a
good idea at all.<br>
<br>
Regards,<br>
Christian.<br>
<br>
Am 12.09.23 um 02:21 schrieb Pan, Xinhui:<br>
> [AMD Official Use Only - General]<br>
><br>
> Oh yep, Pinned BO is moved to other LRU
list, So eviction fails because of other reason.<br>
> I will change the comments in the patch.<br>
> The problem is eviction fails as many
reasons, say, BO is locked.<br>
> ASAIK, kfd will stop the queues and flush
some evict/restore work in its suspend callback.
SO the first eviction before kfd callback likely
fails.<br>
><br>
> -----Original Message-----<br>
> From: Christian König <a
href="mailto:ckoenig.leichtzumerken@gmail.com"
moz-do-not-send="true"><ckoenig.leichtzumerken@gmail.com></a><br>
> Sent: Friday, September 8, 2023 2:49 PM<br>
> To: Pan, Xinhui <a
href="mailto:Xinhui.Pan@amd.com"
moz-do-not-send="true"><Xinhui.Pan@amd.com></a>;
<a href="mailto:amd-gfx@lists.freedesktop.org"
moz-do-not-send="true"
class="moz-txt-link-freetext">
amd-gfx@lists.freedesktop.org</a><br>
> Cc: Deucher, Alexander <a
href="mailto:Alexander.Deucher@amd.com"
moz-do-not-send="true"><Alexander.Deucher@amd.com></a>;
Koenig, Christian
<a href="mailto:Christian.Koenig@amd.com"
moz-do-not-send="true"><Christian.Koenig@amd.com></a>;
Fan, Shikang
<a href="mailto:Shikang.Fan@amd.com"
moz-do-not-send="true"><Shikang.Fan@amd.com></a><br>
> Subject: Re: [PATCH] drm/amdgpu: Ignore
first evction failure during suspend<br>
><br>
> Am 08.09.23 um 05:39 schrieb xinhui pan:<br>
>> Some BOs might be pinned. So the first
eviction's failure will abort<br>
>> the suspend sequence. These pinned BOs
will be unpined afterwards<br>
>> during suspend.<br>
> That doesn't make much sense since pinned
BOs don't cause eviction failure here.<br>
><br>
> What exactly is the error code you see?<br>
><br>
> Christian.<br>
><br>
>> Actaully it has evicted most BOs, so
that should stil work fine in<br>
>> sriov full access mode.<br>
>><br>
>> Fixes: 47ea20762bb7 ("drm/amdgpu: Add
an extra evict_resource call<br>
>> during device_suspend.")<br>
>> Signed-off-by: xinhui pan <a
href="mailto:xinhui.pan@amd.com"
moz-do-not-send="true"><xinhui.pan@amd.com></a><br>
>> ---<br>
>>
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 9
+++++----<br>
>> 1 file changed, 5 insertions(+), 4
deletions(-)<br>
>><br>
>> diff --git
a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
>>
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
>> index 5c0e2b766026..39af526cdbbe 100644<br>
>> ---
a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
>> +++
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c<br>
>> @@ -4148,10 +4148,11 @@ int
amdgpu_device_suspend(struct drm_device<br>
>> *dev, bool fbcon)<br>
>><br>
>> adev->in_suspend = true;<br>
>><br>
>> - /* Evict the majority of BOs
before grabbing the full access */<br>
>> - r =
amdgpu_device_evict_resources(adev);<br>
>> - if (r)<br>
>> - return r;<br>
>> + /* Try to evict the majority of
BOs before grabbing the full access<br>
>> + * Ignore the ret val at first
place as we will unpin some BOs if any<br>
>> + * afterwards.<br>
>> + */<br>
>> +
(void)amdgpu_device_evict_resources(adev);<br>
>><br>
>> if (amdgpu_sriov_vf(adev)) {<br>
>>
amdgpu_virt_fini_data_exchange(adev);<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</blockquote>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<br>
</body>
</html>