https://bugzilla.kernel.org/show_bug.cgi?id=204241
Bug ID: 204241 Summary: amdgpu fails to resume from suspend Product: Drivers Version: 2.5 Kernel Version: 5.2.1-arch1-1-ARCH Hardware: Intel OS: Linux Tree: Mainline Status: NEW Severity: normal Priority: P1 Component: Video(DRI - non Intel) Assignee: drivers_video-dri@kernel-bugs.osdl.org Reporter: kitaev@gmail.com Regression: No
Created attachment 283863 --> https://bugzilla.kernel.org/attachment.cgi?id=283863&action=edit dmesg
Computer fails to resume from suspend.
From the logs it looks like AMDGPU fails to resume.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #1 from Alexander Kitaev (kitaev@gmail.com) --- Created attachment 283865 --> https://bugzilla.kernel.org/attachment.cgi?id=283865&action=edit lspci
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Andrey Grodzovsky (andrey.grodzovsky@amd.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |andrey.grodzovsky@amd.com
--- Comment #2 from Andrey Grodzovsky (andrey.grodzovsky@amd.com) --- Can you post full dmesg log from boot, what card are you using ?
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #3 from Andrey Grodzovsky (andrey.grodzovsky@amd.com) --- OK, checked lspci and it's Ellsmere... Never mind.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #4 from Andrey Grodzovsky (andrey.grodzovsky@amd.com) --- I tried to reproduce it with a kernel which is just a few commits different then this one - https://cgit.freedesktop.org/~agd5f/linux/log/?h=drm-next
I tried with X enabled and in FB console. Was able to suspend and resume with no errors.
I would suggest to build your kernel from the branch above and see if it helps. Also please post your FW info using this command cat /sys/kernel/debug/dri/0/amdgpu_firmware_info
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Andreas Jackisch (andreas.jackisch@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |andreas.jackisch@gmail.com
--- Comment #5 from Andreas Jackisch (andreas.jackisch@gmail.com) --- The same issue started to hit me on gentoo when switching from 5.1.5-gentoo to 5.2.5-gentoo. I reverted back to latest 5.1.21-gentoo and the issue did not come up again. The failure on resume happens every after 5...20 attempts. I'll add message logs, lspci and firmware info.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #6 from Andreas Jackisch (andreas.jackisch@gmail.com) --- Created attachment 284411 --> https://bugzilla.kernel.org/attachment.cgi?id=284411&action=edit var_log_messages for amdgpu_ERROR
search fro "amdgpu" to see it fail after resume
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #7 from Andreas Jackisch (andreas.jackisch@gmail.com) --- Created attachment 284413 --> https://bugzilla.kernel.org/attachment.cgi?id=284413&action=edit lspci from ryzen system
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #8 from Andreas Jackisch (andreas.jackisch@gmail.com) --- Created attachment 284415 --> https://bugzilla.kernel.org/attachment.cgi?id=284415&action=edit amdgpu firmware from ryzen system
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #9 from Andrey Grodzovsky (andrey.grodzovsky@amd.com) --- I was able to reproduce.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #10 from Andrey Grodzovsky (andrey.grodzovsky@amd.com) --- Created attachment 284445 --> https://bugzilla.kernel.org/attachment.cgi?id=284445&action=edit resume_failure.log
The kernel OOPS is just a result of previous GFX ring test failure. Attached log from UMR shows gfx ring is hang around (or right after) first PKT3_SET_CONTEXT_REG because latest PFP_HEADER_DUMP shows 0xc0d46900, this points to possibly that some of the payload within SET_CONTEXT_REG (in gfx_v8_0_get_csb_buffer) causes hang and later this results in ring test failure.
Alex Deucher - Any idea how to confirm this ?
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #11 from Alex Deucher (alexdeucher@gmail.com) --- Does this patch fix it? https://cgit.freedesktop.org/drm/drm/commit/?h=drm-fixes&id=72cda9bb5e21...
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #12 from Andreas Jackisch (andreas.jackisch@gmail.com) --- Created attachment 284807 --> https://bugzilla.kernel.org/attachment.cgi?id=284807&action=edit var_log_meesages_5_2_11
I tested w/ kernel 5.2.11 as it contains the referenced patch "drm/amdgpu: pin the csb buffer on hw init for gfx v8". However, the system did not resume properly as before. This was on the 3rd attempt after almost 24 hours in S3. Reverted back to 5.1.21
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #13 from Andreas Jackisch (andreas.jackisch@gmail.com) --- Created attachment 285079 --> https://bugzilla.kernel.org/attachment.cgi?id=285079&action=edit /var/log/messages w/ kernel 5.3.0-gentoo
As there was no success w/ 5.2.x at all I tested 5.3.0. However, the system did not resume after the 2nd attempt with a comparable failure message.
amdgpu 0000:06:00.0: [drm:amdgpu_ring_test_helper] *ERROR* ring sdma0 test failed (-110)
This is slightly different from 5.2.x where it was
amdgpu 0000:06:00.0: [drm:amdgpu_ring_test_helper] *ERROR* ring gfx test failed (-110)
but the result seems to be the same.
I'm not sure whether anybody is working on this or the bug-opener still sees the issue. As latest kernel series 5.1.x is somehow outdated now I will revert to 4.19.x LTS. If there is any hint or advise what I can do to help please let me know.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Ahzo@tutanota.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |Ahzo@tutanota.com
--- Comment #14 from Ahzo@tutanota.com --- Created attachment 285349 --> https://bugzilla.kernel.org/attachment.cgi?id=285349&action=edit Patch to prevent frequent resume failures
While this issue happens rather randomly, it can be quite reliably reproduced on linux 5.2 and later by performing successive suspend-resume cycles. Usually the error occurs after less than 10 cycles, but occasionally only after more than 20. Thus one can use the following command to reproduce it almost certainly: $ for i in $(seq 30); do sudo rtcwake -m mem -s 5; sleep 15; done
A bisection using this method lead to: commit 533aed278afeaa68bb5d0600856ab02268cfa3b8 Author: Andrey Grodzovsky andrey.grodzovsky@amd.com Date: Wed Mar 6 16:16:28 2019 -0500
drm/amdgpu: Move IB pool init and fini v2
Problem: Using SDMA for TLB invalidation in certain ASICs exposed a problem of IB pool not being ready while SDMA already up on Init and already shutt down while SDMA still running on Fini. This caused IB allocation failure. Temproary fix was commited into a bringup branch but this is the generic fix.
Fix: Init IB pool rigth after GMC is ready but before SDMA is ready. Do th opposite for Fini.
v2: Remove restriction on SDMA early init and move amdgpu_ib_pool_fini
Reviewed-by: Christian König christian.koenig@amd.com Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Signed-off-by: Alex Deucher alexander.deucher@amd.com
Reverting this commit makes the problem unreproducible with above command.
Another way to prevent these frequent resume failures, while preserving the intention of this commit, is to simply call amdgpu_ib_pool_init directly after calling amdgpu_ucode_create_bo instead of directly before that. Attached is a patch doing it that way.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #15 from Andreas Jackisch (andreas.jackisch@gmail.com) --- (In reply to Ahzo from comment #14)
Created attachment 285349 [details] Patch to prevent frequent resume failures .... Another way to prevent these frequent resume failures, while preserving the intention of this commit, is to simply call amdgpu_ib_pool_init directly after calling amdgpu_ucode_create_bo instead of directly before that. Attached is a patch doing it that way.
I applied the patch above to 5.3.2-gentoo. All 30 Suspend/Resume cycles using rtcwake and a couple of manual cycles went OK.
I'll continue to use this setup and will report if it fails again or is still OK after one week.
Thx for bisecting this issue and providing this fix as I assume it took some time.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #16 from me@cschwarz.com --- Can confirm the patch 'drm/amdgpu: Move IB pool init after ucode bo creation' fixed the issue for me (96h and counting, failure normally within 24h, with ~2 suspend/resume cycles per day).
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #17 from Alex Deucher (alexdeucher@gmail.com) --- (In reply to Ahzo from comment #14)
Another way to prevent these frequent resume failures, while preserving the intention of this commit, is to simply call amdgpu_ib_pool_init directly after calling amdgpu_ucode_create_bo instead of directly before that. Attached is a patch doing it that way.
I'm not sure I understand why the patch helps. You are just changing the order of two memory allocations. The order shouldn't matter.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #18 from Michel Dänzer (michel@daenzer.net) --- (In reply to Alex Deucher from comment #17)
I'm not sure I understand why the patch helps. You are just changing the order of two memory allocations. The order shouldn't matter.
My guess would be that the exact location of the ucode BO matters somehow.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #19 from me@cschwarz.com --- Just had the first (but different kind of) crash since applying the patch on top of 5.3.2, but didn't have kdump configured: The system woke, everything seemed to work for about 30s, then the screen went black and the machine rebooted.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Ahzo@tutanota.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #285349|0 |1 is obsolete| |
--- Comment #20 from Ahzo@tutanota.com --- Created attachment 285469 --> https://bugzilla.kernel.org/attachment.cgi?id=285469&action=edit Patch to fix the resume failures
(In reply to Alex Deucher from comment #17)
I'm not sure I understand why the patch helps. You are just changing the order of two memory allocations. The order shouldn't matter.
My hypothesis is that the order here is not the root cause of the problem, but rather affects the likelihood of that manifesting itself. This is based on the fact that I have seen a resume failure typical for this bug on linux 5.0 once, but I'm unable to reproduce it with that version.
As commit 533aed278afe apparently makes the failures much more likely to happen, it provides an opportunity to debug this further by backporting it to older linux versions. Doing that for versions down to linux 4.15 exposes the resume failures, but not on linux 4.14.
A bisection between these two, while backporting 533aed278afe on every step, lead to commit 2a91f272e34c, which failed to boot and thus had to be skipped, and: commit e0128efb08b3d628d767ec8578e77cdd7ecc8f81 Author: James Zhu James.Zhu@amd.com Date: Fri Sep 29 16:42:27 2017 -0400
drm/amdgpu: add uvd enc ib test
Generate create/destroy messages to test UVD encode indirect buffer function. And enable UVD encode IB test during device initialization.
Signed-off-by: James Zhu James.Zhu@amd.com Reviewed-and-Tested-by: Leo Liu leo.liu@amd.com Reviewed-by: Christian König christian.koenig@amd.com Signed-off-by: Alex Deucher alexander.deucher@amd.com
This looks like a likely root cause. Indeed, adding 'return 0;' at the beginning of uvd_v6_0_enc_ring_test_ib makes the problem unreproducible, even on the latest linux 5.4-rc2.
Comparing with amdgpu_uvd_get_{create,destroy}_msg shows that these use 0 as dummy GPU pointer, while uvd_v6_0_enc_get_{create,destroy}_msg use a real GPU memory address. Changing them to also use 0 as dummy pointer, as is done in the attached patch, actually fixes the resume failures.
Maybe a similar change should also be made for UVD 7.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #21 from Ahzo@tutanota.com --- Created attachment 285471 --> https://bugzilla.kernel.org/attachment.cgi?id=285471&action=edit Patch to prevent kernel NULL pointer dereferences
By the way, some of the kernel NULL pointer dereferences, that can happen after a resume failure, also happen always on shutdown: RIP: 0010:build_audio_output.isra.0+0x97/0x110 [amdgpu] RIP: 0010:enable_link_dp+0x186/0x300 [amdgpu]
Attached patch prevents them.
Note that these oopses are difficult to notice on shutdown, because they only leave traces in /sys/fs/pstore, not on the disk, as they happen after unmounting.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #22 from Alex Deucher (alexdeucher@gmail.com) --- Created attachment 285473 --> https://bugzilla.kernel.org/attachment.cgi?id=285473&action=edit possible fix uvd6
Nice work. I think the attached patch should fix it.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #23 from Alex Deucher (alexdeucher@gmail.com) --- Created attachment 285475 --> https://bugzilla.kernel.org/attachment.cgi?id=285475&action=edit possible fix uvd7
Same fix for uvd7.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #24 from Ahzo@tutanota.com --- (In reply to Alex Deucher from comment #22)
Created attachment 285473 [details] possible fix uvd6
Nice work. I think the attached patch should fix it.
Thanks for finding the correct solution. I can confirm that the patch for uvd6 works. The one for uvd7 also looks good, but I don't have the hardware to test it. Furthermore, I think vcn also needs a similar change. I'm not sure about vce, as that uses 'ib_size_dw = 1024' thus allocating a much larger buffer.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #25 from me@cschwarz.com --- If it is of any help: I would be willing to test any of the more recent patches.
Hardware: - Radeon RX 550 - Ryzen 1700X
The first patch by Ahzo@ already worked for me: 5.3.2 with "drm/amdgpu: Move IB pool init after ucode bo creation"
What other patches should I test with which kernel version? Please provide Bugzilla attachment numbers.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #26 from Ahzo@tutanota.com --- You can test Alex Deucher's uvd6 patch (attachment 285473), which is the proper fix for your RX 550. Testing on linux 5.3 is fine, as this patch should fix the problem on any affected version.
The patch you tested previously just makes the problem unlikely to cause resume failures, but it doesn't fix the root cause of overwriting random GPU memory, so it might still cause random issues.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #27 from Andreas Jackisch (andreas.jackisch@gmail.com) --- In brief - the patch "0001-drm-amdgpu-uvd6-fix-allocation-size-in-enc-ring-test.patch" didn't work for me. After about 10 suspend/resume cycles the typical issue occurred again and I had to SysRq the system.
Status, all gentoo kernels: 5.1.x OK 4.19.74 OK 5.2.x FAIL 5.3.0 FAIL 5.3.2 w/ patch from comment#14 OK 5.3.6 FAIL 5.3.6 w/ patch 0001-drm-amdgpu-uvd6-fix-allocation-size-in-enc-ring-test FAIL 5.3.6 w/ patch 0001-drm-amdgpu-uvd6-use-0-as-dummy-pointer-in-enc-ring-t OK
The last setup has seen 30+ suspend/resume cycles. I'll continue to use this.
So, to me it looks like that increasing the allocation did not help but assigning 0 to the dummy pointer did.
My hardware is comparable to the one listed in comment#25 - Radeon RX550 - Ryzen 1700
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #28 from Ahzo@tutanota.com --- (In reply to Andreas Jackisch from comment #27)
In brief - the patch "0001-drm-amdgpu-uvd6-fix-allocation-size-in-enc-ring-test.patch" didn't work for me. After about 10 suspend/resume cycles the typical issue occurred again and I had to SysRq the system.
Indeed, the 0001-drm-amdgpu-uvd6-fix-allocation-size-in-enc-ring-test patch (attachement 285473) doesn't work. Apparently I got (un)lucky enough that it survived 30 suspend/resume cycles, but testing it again, it failed.
On the other hand, the 0001-drm-amdgpu-uvd6-use-0-as-dummy-pointer-in-enc-ring-t patch (attachement 285469) survived 100 cycles.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Alex Deucher (alexdeucher@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #285473|0 |1 is obsolete| |
--- Comment #29 from Alex Deucher (alexdeucher@gmail.com) --- Created attachment 285507 --> https://bugzilla.kernel.org/attachment.cgi?id=285507&action=edit possible fix for uvd6
The session info is 128K according to mesa.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Alex Deucher (alexdeucher@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #285475|0 |1 is obsolete| |
--- Comment #30 from Alex Deucher (alexdeucher@gmail.com) --- Created attachment 285509 --> https://bugzilla.kernel.org/attachment.cgi?id=285509&action=edit possible fix uvd7
Updated patch for uvd7
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #31 from Alex Deucher (alexdeucher@gmail.com) --- Created attachment 285511 --> https://bugzilla.kernel.org/attachment.cgi?id=285511&action=edit possible fix for vcn
Same fix for vcn.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #32 from me@cschwarz.com --- @Alex: Didn't have a crash with the old uvd6 patch (attachment 285473) so far, but apparently I am just lucky.
Which patch (series?) should I test now?
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #33 from Alex Deucher (alexdeucher@gmail.com) --- (In reply to me from comment #32)
@Alex: Didn't have a crash with the old uvd6 patch (attachment 285473 [details]) so far, but apparently I am just lucky.
Which patch (series?) should I test now?
Please try attachment 285507.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #34 from Ahzo@tutanota.com --- (In reply to Alex Deucher from comment #29)
Created attachment 285507 [details] possible fix for uvd6
The session info is 128K according to mesa.
This version of the patch didn't fail for 100 suspend/resume cycles, so I think it actually fixes the problem.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #35 from Andreas Jackisch (andreas.jackisch@gmail.com) ---
(In reply to Ahzo from comment #34)
(In reply to Alex Deucher from comment #29)
Created attachment 285507 [details] possible fix for uvd6
The session info is 128K according to mesa.
This version of the patch didn't fail for 100 suspend/resume cycles, so I think it actually fixes the problem.
I can confirm that the patch seems to work OK. 30+ suspend/resume cycles so far where it normally fails after 10 cycles.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Mario (kernel@catmail.app) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |kernel@catmail.app
--- Comment #36 from Mario (kernel@catmail.app) --- I can also confirm this patch (285507) fixed the problem on Arch Linux 5.3.7.
The stock kernel failed after ~5 sleep-wake cycles. Patched kernel was able to survive the complete 30 cycles:
```for i in $(seq 30); do sudo rtcwake -m mem -s 5; sleep 15; done```
Thanks for the patch. I also suspect that bug 204965 is a duplicate of this one.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
David (dav.per@gmx.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |dav.per@gmx.com
--- Comment #37 from David (dav.per@gmx.com) --- *** Bug 204965 has been marked as a duplicate of this bug. ***
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Andrew Hutchings (andrew@linuxjedi.co.uk) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |andrew@linuxjedi.co.uk
--- Comment #38 from Andrew Hutchings (andrew@linuxjedi.co.uk) --- Also confirmed Alex Deucher's patches work great for me, patched Fedora 31 kernel 5.3.7 on a ThinkPad T495 Ryzen 7 PRO 3700U with a Vega 10 GPU (vcn).
Many thanks!
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #39 from me@cschwarz.com --- (In reply to Alex Deucher from comment #33)
(In reply to me from comment #32)
@Alex: Didn't have a crash with the old uvd6 patch (attachment 285473
[details]
[details]) so far, but apparently I am just lucky.
Which patch (series?) should I test now?
Please try attachment 285507 [details].
Can confirm this patch works, 40 days uptime, _many_ suspend-resume cycles, no problems.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Frans Skarman (frans.skarman@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |frans.skarman@gmail.com
--- Comment #40 from Frans Skarman (frans.skarman@gmail.com) --- This patch did not solve the issue for me, or rather, the arch build system says the patch is already applied in 5.4.2-arch.
Suspend consistently doesn't work, and the first issue reported by journalctl is the aformentioned amdgpu (-110) error.
This is with a ryzen 7 3800x and rx 580
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Ulf Winkelvos (ulf@winkelvos.de) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |ulf@winkelvos.de
--- Comment #41 from Ulf Winkelvos (ulf@winkelvos.de) --- Created attachment 286215 --> https://bugzilla.kernel.org/attachment.cgi?id=286215&action=edit suspend crash on Lenovo Thinkpad T495 Kernel 5.3.13-arch1-1
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #42 from Ulf Winkelvos (ulf@winkelvos.de) --- On my System Lenovo ThinkPad T495 (model 20NKS01Y00) the crashes still happen on every 1st to 4th suspend (see above log).
--- amdgpu 0000:06:00.0: [drm:amdgpu_ib_ring_tests [amdgpu]] *ERROR* IB test failed on gfx (-110). ---
I found out though that if i disable my fingerprint reader, aswell as the smartcard reader in bios the crashes do not occour anymore:
--- -Bus 003 Device 006: ID 06cb:00bd Synaptics, Inc. -Bus 003 Device 005: ID 058f:9540 Alcor Micro Corp. AU9540 Smartcard Reader ---
https://bugzilla.kernel.org/show_bug.cgi?id=204241
crab2313@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |crab2313@gmail.com
--- Comment #43 from crab2313@gmail.com --- Same problem with my Thinkpad x395 (model 20NL000YCD). The system refused to suspend consistently and showed a blurred screen. Also, the LED on power button do not turn off.
The issue still exist when I disable fingerprint reader and SD card reader in bios.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #44 from crab2313@gmail.com --- Created attachment 286253 --> https://bugzilla.kernel.org/attachment.cgi?id=286253&action=edit log of x395 when suspend
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #45 from crab2313@gmail.com --- Kernel 5.4.2 and kernel 5.3 is affected. I switch to kernel 5.2.19 and do not have this issue.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #46 from Alex Deucher (alexdeucher@gmail.com) --- Can you bisect? It sounds like you may be experiencing a different issue.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #47 from crab2313@gmail.com --- @Alex Deucher
Unfortunately, I discovered switch to 5.2.19 just lower the possibility of my issue. I think bisect can not find the root cause.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
muncrief (rmuncrief@humanavance.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |rmuncrief@humanavance.com
--- Comment #48 from muncrief (rmuncrief@humanavance.com) --- I have an R9-390 that just started having the resume from suspend problem as of 5.5-rc1. And I just tested 5.5-rc2 and the problem persists.
The problem looks exactly the same as the one that plagued the R9-390 starting with the 4.20 kernel, but was fixed a few releases later.
My system goes into suspend mode normally, but when resuming my monitor says "Signal not recognized" and I have to SSH into my system and reboot it.
I'm running Manjaro with Mesa 19.2.7 amdgpu, and the last working kernel is 5.4.2. So something in the new 5.5 amdgpu has borked the R9-390 again.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #49 from Alex Deucher (alexdeucher@gmail.com) --- (In reply to muncrief from comment #48)
I have an R9-390 that just started having the resume from suspend problem as of 5.5-rc1. And I just tested 5.5-rc2 and the problem persists.
The problem looks exactly the same as the one that plagued the R9-390 starting with the 4.20 kernel, but was fixed a few releases later.
My system goes into suspend mode normally, but when resuming my monitor says "Signal not recognized" and I have to SSH into my system and reboot it.
I'm running Manjaro with Mesa 19.2.7 amdgpu, and the last working kernel is 5.4.2. So something in the new 5.5 amdgpu has borked the R9-390 again.
This sounds like a different issue, please file a different ticket.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #50 from Ulf Winkelvos (ulf@winkelvos.de) --- I tried to bisect this issue in the past days, but it is almost impossible to track it down, as it is so hard to reproduce it reliably. It seems that 5.2 is "better", the close the commits go to 5.3 it gets "worse". Now all of a sudden 5.4.3-arch1-1 is completely stable so far... I am going to create a new bug, whenever this comes back.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #51 from Alexander Jones (acjones8@hawaii.edu) --- For what it's worth, I believe I might be suffering from this bug. I have a ThinkPad A275, with an AMD A12 9800B CPU and an R7 integrated GPU, and I can reliably produce a crash on suspend every single time. It produces an image like this when it wakes up: https://imgur.com/tKAxlI7
As you can see, a complete garbled mess. X11 becomes completely unresponsive; I can't quit it, switch to a VT, or do anything whatsoever, only a hard reset fixes it. The screen glitchiness does seem to flicker and slightly change while mashing buttons though. Other aspects of the computer work fine though; the CPU fan maintains the same speed, the power LED blinks normally, the dot on the i on the back of the lid pulses like normal, and I can still change the keyboard's backlight with no problems.
I'm running OpenSUSE Tumbleweed at the moment. With kernel 5.2.X, I never had any crashes whatsoever, but once Tumbleweed updated to 5.3 or 5.4, it will fail every single time to resume. I'm currently running 5.5.4.1 and the issue is still here. I don't have any kernel hacking or debugging experience, but I'm willing to upload any logs that might prove helpful, if you can tell me which ones those might be.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #52 from dimitris@gmail.com --- This is a shot in the dark/cargo culting it, but in case it helps:
I had a very similar problem on a T495 (Ryzen 3700U), running Fedora 31, which resolved itself when the 5.4 series was available in Fedora.
Before 5.4 was available, I came across reports linking this to a USB controller of all things, like https://www.mail-archive.com/debian-kernel@lists.debian.org/msg116563.html.
In my case the cuprit was:
06:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
so I started removing the device from the PCI tree before suspend using /sys/bus/pci/devices/0000:06:00.4/remove and rescanning the PCI bus on resume. First manually and later though a systemd hook. That worked around the problem until 5.4 "fixed" this.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #53 from Alexander Jones (acjones8@hawaii.edu) --- (In reply to dimitris from comment #52)
This is a shot in the dark/cargo culting it, but in case it helps:
I had a very similar problem on a T495 (Ryzen 3700U), running Fedora 31, which resolved itself when the 5.4 series was available in Fedora.
Before 5.4 was available, I came across reports linking this to a USB controller of all things, like https://www.mail-archive.com/debian-kernel@lists.debian.org/msg116563.html.
In my case the cuprit was:
06:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
so I started removing the device from the PCI tree before suspend using /sys/bus/pci/devices/0000:06:00.4/remove and rescanning the PCI bus on resume. First manually and later though a systemd hook. That worked around the problem until 5.4 "fixed" this.
Thank you for the suggestion! I tried that out on my ThinkPad, disabling all of my USB devices just in case. They are:
00:10.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB XHCI Controller (rev 20) 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller (rev 49) 01:00.4 USB controller: Realtek Semiconductor Co., Ltd. Device 816d (rev 0e)
Unfortunately, this didn't fix the suspend issue, I still get the glitchy screen. On a whim, I tried to disable Bluetooth in the BIOS, as well as the Fingerprint Scanner and TPM chip, but that also didn't have any affect. Coincidentally, I DID hear Kmail pop a notification after resuming, so it seems it's not as dead as I thought, the kernel and even the userland seem to still work then. Must be AMDGPU or something else in the graphics stack that dies, since I can't switch to a VT and suspending while in a VT doesn't work either, and results in the same glitched out mess.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #54 from Bjoern Franke (bjo@nord-west.org) --- @Alexander Jones:
Regarding the garbled screen after resume, there's another bugreport: https://bugzilla.kernel.org/show_bug.cgi?id=206393
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #55 from Alexander Jones (acjones8@hawaii.edu) --- Thank you very much for that link, Mr. Franke! That bug report much more closely approximates my situation, down to a T. I tried the older kernel suggestion listed there, I still have a backup copy of 5.4.10 (but not anything earlier), and it works perfectly again! That solves my problem in the short term with Tumbleweed. I'm not sure then if it's related to this AMDGPU bug and just manifesting itself differently, or if they're actually different bugs, but I'll switch to over to that thread then. Thank you once again!
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #56 from Frans Skarman (frans.skarman@gmail.com) --- I experienced this issue (black screen after resuming from suspend) for a while on my ryzen 3800x + rx 580 setup. Same issues happened with every kernel i tried. Eventually, I figured out that a BIOS update fixed the issues (this was an MSI B450 tomahawk max).
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Jordan Maris (jman6495@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jman6495@gmail.com
--- Comment #57 from Jordan Maris (jman6495@gmail.com) --- I'm also experiencing this issue on a HP Envy 13 x360 with the Ryzen 3500U APU. Has anyone found any potential solutions ?
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Jaya Balan Aaron (bucket.size@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |bucket.size@gmail.com
--- Comment #58 from Jaya Balan Aaron (bucket.size@gmail.com) --- Created attachment 288507 --> https://bugzilla.kernel.org/attachment.cgi?id=288507&action=edit arch linux 5.6 resolution with kernel params
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #59 from Jaya Balan Aaron (bucket.size@gmail.com) --- Comment on attachment 288507 --> https://bugzilla.kernel.org/attachment.cgi?id=288507 arch linux 5.6 resolution with kernel params
Hi,
Using arch linux kernel 5.5zen, 5.6. Not sure if it's a solution but, interesting to note.
With 5.6, with kernel params 'amd_iommu=on iommu=pt', able to suspend/resume correctly 10/10 times. Without the params resume hanged with a blank and backlit screen 2/2 times.
With 5.5zen, even with the same kernel params, resume hanged 2/2 times.
Reason for the kernel params is that I was trying to set up gpu passthrough with kvm.
Suspend resumes immediately sometimes, but I think that's because of mis-configured, keyboard/mouse/usb wake triggers.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #60 from Jordan Maris (jman6495@gmail.com) --- Created attachment 289129 --> https://bugzilla.kernel.org/attachment.cgi?id=289129&action=edit dmesg log on suspend
https://bugzilla.kernel.org/show_bug.cgi?id=204241
igor@sonce.de changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |igor@sonce.de
--- Comment #61 from igor@sonce.de --- (In reply to Ulf Winkelvos from comment #50)
I tried to bisect this issue in the past days, but it is almost impossible to track it down, as it is so hard to reproduce it reliably. It seems that 5.2 is "better", the close the commits go to 5.3 it gets "worse". Now all of a sudden 5.4.3-arch1-1 is completely stable so far... I am going to create a new bug, whenever this comes back.
Thank you, You saved my day. Switched from 5.4.0 to 5.4.3 and now I am able to properly suspend and resume. With 5.4.0 the system crashed and reset.
keep save and healthy. By Igor
https://bugzilla.kernel.org/show_bug.cgi?id=204241
poinck (andre@poinck.de) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |andre@poinck.de
--- Comment #62 from poinck (andre@poinck.de) --- I am having the same issue with:
Platform: Linux-5.4.38-gentoo-x86_64-Intel-R-_Core-TM-i5-3570K_CPU@_3.40GHz-with-gentoo-2.6, 64bit Graphics: 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev cf) DE: Gnome 3.34.4
Steps to repoduce: - start qutebrowser (uses Qt-5.14.2) under Gnome - hibernate - system freezes immediatly (or just blank screen and remotely still available) or eventually blanks after resume. - restart or hard reset neccessary
Workarround: - stop qutebrowser before hibernating - resume works and I can login normally and resume the session
https://bugzilla.kernel.org/show_bug.cgi?id=204241
felipejfc (fjfcavalcanti@gmail.com) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |fjfcavalcanti@gmail.com
--- Comment #63 from felipejfc (fjfcavalcanti@gmail.com) --- I'm having the same issue with an AMD RX5700 and kernel version 5.7.9-1 on manjaro linux.
for me adding kernel params 'amd_iommu=on iommu=pt' didn't solve the problem. graphics won't turn on so monitor just keeps blinking
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #64 from felipejfc (fjfcavalcanti@gmail.com) --- (In reply to felipejfc from comment #63)
I'm having the same issue with an AMD RX5700 and kernel version 5.7.9-1 on manjaro linux.
for me adding kernel params 'amd_iommu=on iommu=pt' didn't solve the problem. graphics won't turn on so monitor just keeps blinking
complementing my last answer, the "fix" that worked for me was to disable IOMMU on BIOS
https://bugzilla.kernel.org/show_bug.cgi?id=204241
waltibaba@protonmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |waltibaba@protonmail.com
--- Comment #65 from waltibaba@protonmail.com --- I'm getting the issue again on 5.8.9-arch2-1 - though compounding it is that suspending fails and it instantly tries to resume with a black screen. Can confirm that it was not present on 5.8.8-arch1-1 before it (that ran with many suspend/resume cycles for a week).
Prime B350M-A R7 1700 Fury X (Fiji)
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #66 from waltibaba@protonmail.com --- Created attachment 292637 --> https://bugzilla.kernel.org/attachment.cgi?id=292637&action=edit dmesg 5.8.9-arch2-1
truncated dmesg logs of resume failures on 5.8.9-arch2-1 3 boot/suspend/resume/fail cycles occurred
https://bugzilla.kernel.org/show_bug.cgi?id=204241
Lahfa Samy (samy@lahfa.xyz) changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |samy@lahfa.xyz
--- Comment #67 from Lahfa Samy (samy@lahfa.xyz) --- I'm having currently this issue on a T495 with a Ryzen 3700U with integrated graphics Vega RX 10 on ArchLinux with ZFS.
Before 5.8.12-arch1-1, I can suspend however right when I resume the system freezes.
I have to hard reset by rebooting using the power button, nothing is present in the journalctl besides systemd saying it did suspend, it's not mentioning something that fails about AMDGPU.
However have seen a call trace in dmesg about the wifi driver (RIP: 0010:iwl_pcie_rx_handle+0x9c7/0xbb0 [iwlwifi]) but this is happening during boot and thus maybe not affecting the suspend process.
The thing is this issue started when I upgraded the kernel from 5.8.11-arch to 5.8.12 but I have also installed AMDGPU (bad timing) and Mesa-git thus I'm not being too sure if the latter is maybe part of the issue or is the very problem of this bug.
I have removed the git packages and installed their stable counterparts also removed the kernel parameters amdgpu.cik_support=1 amdgpu.sk_support=1 radeon.sk_support=0 radeon.cik_support=0 and I'll be doing some tests and reporting if I find a way to mitigate the issue.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #68 from Robert M. Muncrief (rmuncrief@humanavance.com) --- Created attachment 292729 --> https://bugzilla.kernel.org/attachment.cgi?id=292729&action=edit Resume fail with RX 580 GPU
I've been having random resume problems form around kernel 5.5, and it persists even up to 5.9-rc6. When this occurs I can still login to SSH and give a reboot command, but though SSH disconnects my computer doesn't reboot and I have to press the reset button.
I have an ASUS Gaming TUF X570 motherboard, R7 3700X CPU, RX 580 GPU, and 16GB of RAM.
The primary error recorded in dmesg is:
[xxxxx.xxxxxx] amdgpu: last message was failed ret is 65535
I've included the part of dmesg beginning with suspend event through the resume failure.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #69 from Lahfa Samy (samy@lahfa.xyz) --- I've got news of a current workaround for my T495 with a Ryzen 7 3700U and a Vega RX 10 on kernel 5.8.12arch, I have disabled the Network card (which means no more WiFi at all) in the BIOS and this has solved the problem of the resuming freeze. This is most likely due to a bug in the driver iwlwifi used by the Intel Wireless AC-9260 network card, I can also confirm that the same bug affects the package linux-lts for ArchLinux 5.4.68-1-lts.
The logs show a watchdog :soft-lockup on CPU#0 stuck for 22s! [irq/87-iwlwifi::979].
Later in the log there is this line : RIP : 0010:iwl_trans_pcie_read32+0x10/0x20 [iwlwifi]
A few more information probably that would help someone make a patch maybe. And finally a call trace.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #70 from Lahfa Samy (samy@lahfa.xyz) --- I've opened a new bug report as the issue is clearly related to networking and the iwlwifi driver and not to the AMDGPU driver in my case. Here is the link to the bug report : https://bugzilla.kernel.org/show_bug.cgi?id=209435
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #71 from Alex Deucher (alexdeucher@gmail.com) --- The original issue reported in this bug was fixed long ago. If you are having issues, please file a new report.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #72 from Robert M. Muncrief (rmuncrief@humanavance.com) --- (In reply to Alex Deucher from comment #71)
The original issue reported in this bug was fixed long ago. If you are having issues, please file a new report.
I just filed a new bug for the resume issue at your request. It's 209457.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #73 from Lahfa Samy (samy@lahfa.xyz) --- (In reply to Robert M. Muncrief from comment #72)
(In reply to Alex Deucher from comment #71)
The original issue reported in this bug was fixed long ago. If you are having issues, please file a new report.
I just filed a new bug for the resume issue at your request. It's 209457.
My issue seems unrelated to your bug report, my suspend/resume freeze issue is related to my Intel Wireless AC9260 not to my AMD Ryzen 7 3700U with integrated graphics Vega RX10.
Disabling the wireless card in the BIOS fixes the suspend/resume problem for my specific configuration (Thinkpad T495 20NK model).
Although your issue seems to be with the AMDGPU driver and related to your graphics card I suppose.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #74 from Robert M. Muncrief (rmuncrief@humanavance.com) --- (In reply to Lahfa Samy from comment #73)
(In reply to Robert M. Muncrief from comment #72)
(In reply to Alex Deucher from comment #71)
The original issue reported in this bug was fixed long ago. If you are having issues, please file a new report.
I just filed a new bug for the resume issue at your request. It's 209457.
My issue seems unrelated to your bug report, my suspend/resume freeze issue is related to my Intel Wireless AC9260 not to my AMD Ryzen 7 3700U with integrated graphics Vega RX10.
Disabling the wireless card in the BIOS fixes the suspend/resume problem for my specific configuration (Thinkpad T495 20NK model).
Although your issue seems to be with the AMDGPU driver and related to your graphics card I suppose.
Yes, I filed a new bug for my issue at https://bugzilla.kernel.org/show_bug.cgi?id=209457.
Hopefully this bug will be closed to avoid further confusion for users, and relieve the hard working developers from our confusion as well :)
https://bugzilla.kernel.org/show_bug.cgi?id=204241
a.geno@libero.it changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |a.geno@libero.it
--- Comment #75 from a.geno@libero.it --- I've this problem too. Still happening. Currently I have the 5.10.7-3 kernel.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #76 from Robert M. Muncrief (rmuncrief@humanavance.com) --- I also continue to have this problem on Arch with kernel 5.10.14.
https://bugzilla.kernel.org/show_bug.cgi?id=204241
--- Comment #77 from Alex Deucher (alexdeucher@gmail.com) --- Please open a new ticket this issue was fixed.
dri-devel@lists.freedesktop.org
-
bugzilla-daemon＠bugzilla.kernel.org