<html> <head> <base href="https://bugs.freedesktop.org/"> </head> <body> <div> <a class="bz_bug_link bz_status_NEW " title="NEW - [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx timeout" href="https://bugs.freedesktop.org/show_bug.cgi?id=110509#c9">Comment # 9</a> on <a class="bz_bug_link bz_status_NEW " title="NEW - [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx timeout" href="https://bugs.freedesktop.org/show_bug.cgi?id=110509">bug 110509</a> from <a class="email" href="mailto:alexdeucher@gmail.com" title="Alex Deucher <alexdeucher@gmail.com>"> Alex Deucher</a> <pre>(In reply to James.Dutton from <a href="show_bug.cgi?id=110509#c8">comment #8</a>) > Thank you for the feedback. > Is there a data sheet somewhere that might help me work out a fix for this. > What I would like is: > 1) A way to scan all the engines and detect which ones have hung. If the gpu scheduler for a queue on a particular engine times out, you can be pretty sure the engine has hung. At that point you can check the current busy status for the block (IP is_idle() callback). > 2) A way to intentionally halt an engine and tidy up. So that the modprobe, > rmmod, modprobe scenario works. hw_fini() IP callback. > 3) data sheet details regarding how to un-hang each engine. > Specifically, in this case the IP block <dm>. Each IP has a soft reset (implemented via the IP soft_reset() callback), but depending on the hang, in some cases, you may have to do a full GPU reset to recover. This is not a hw hang, it's a sw deadlock. > > Maybe that is not possible, and (I think you are hinting at it), one cannot > reset an individual IP block. So the approach is to suspend the card, and > then do a full reset of the entire card, then resume. All asics support full GPU reset which is implemented via the SOC level amdgpu_asic_funcs reset() callback. > > I think a different suspend process would be better. > We have a for_each within the suspend code. The output of that code should > not be a single error code, but instead an array indicating the current > state of each engine (running/hung), the intended state and status of > whether the intention worked or failed. If the loop through the for_each, it > could compare the current state and intended state, and attempt to reach the > intended state, and report an error code for each engine. Then the code to > achieve the transition can been different depending on the current -> > intended transition. > i.e. code for running -> suspended, can be different than code for hung -> > suspended. The code already needs to know which engines are enabled/disabled > (Vega 56 vs Vega 64) We don't really care of the suspend fails or not. See amdgpu_device_gpu_recover() for the full sequence. > > I can hang this IP block <dm> at will. I have 2 games that hang it within > seconds of starting. There was a deadlock in the dm code which has been fixed. Please try a new code base. e.g., <a href="https://cgit.freedesktop.org/~agd5f/linux/log/?h=amd-staging-drm-next">https://cgit.freedesktop.org/~agd5f/linux/log/?h=amd-staging-drm-next</a> <a href="https://cgit.freedesktop.org/~agd5f/linux/log/?h=drm-next-5.2-wip">https://cgit.freedesktop.org/~agd5f/linux/log/?h=drm-next-5.2-wip</a></pre> </div> <hr> You are receiving this mail because: <ul> <li>You are the assignee for the bug.</li> </ul> </body> </html>