[Intel-gfx] [PATCH 0/3] Per Engine hang detection and recovery

[Daniel Vetter]

On Mon, Nov 11, 2013 at 03:49:01PM +0000, Siluvery, Arun wrote:
> On Mon, 2013-11-11 at 16:31 +0100, Daniel Vetter wrote:
> > On Mon, Nov 11, 2013 at 02:58:31PM +0000, Siluvery, Arun wrote:
> > > From: "Siluvery, Arun" <arun.siluvery at intel.com>
> > > 
> > > This patchset contains changes for Timeout detection and recovery (TDR) which
> > > provides per-engine hang detection and recovery.
> > > The current driver performs full gpu reset in case of a hang, TDR attempts to
> > > only reset the engine that is hung and it falls back to full reset if it fails.
> > > 
> > > Full GPU reset can leave the system in a state where the display updates
> > > intermittently and possibly lock-up depending on the work load at the time of
> > > hang. TDR can help recover the system in those case thus increasing the stability.
> > 
> > Are these hw lockups you've seen with full gpu reset or just kernel
> > deadlocks? If it's the latter we've recently (re-)fixed a bunch of those,
> > and if there are new ones we definitely want to fix them and add testcases
> > to igt. So if you could share some of these hangs and their
> > analysis/testcases that's be very interesting.
> > 
> > That's of course on top of any other reset improvements.
> 
> I think these are kernel lockups, unfortunately when this happens there
> is no response from the kernel, sending break is also not helping. I
> will try to get more details on this.

Enabling lockdep and making sure you have the stuck task warning enabled
occasionally helps to get something out of the system. Is the entire box
dead or just everything related to gfx?

> > > The changes are split in multiple patches.
> > > 1. Ring utility functions to save/restore context, reset ring etc
> > > 2. TDR hang detection logic and error recovery function
> > > 3. Debugfs changes to export TDR statistics.
> > > 
> > > I have tested these changes on drm-intel-nightly with simple test which
> > > inserts a bad batch buffer on the specific to trigger a hang. TDR logic
> > > then detects this and recovers from it by skipping the bad batch.
> > 
> > I want this testcase (as a patch to igt).
> 
> ok, I will send it to the mailing list.

Thanks.

> > > Please review and give your comments.
> > 
> > I'll try to have a look later this week, atm still busy with bdw
> > upstreaming. One more meta-comment though: Something with your git setup
> > seems to be broken, the patches don't have in-reply-to headers pointing at
> > this cover letter and hence the threading is a bit broken.
> 
> ok thanks.
> yes my mistake I missed an option while generating the patches.
> Do you suggest resending all patches again?

No need, was just a quick reminder for next time around. Without threading
patch groups are harder to find, especially when a bigger discussion
ensued.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch