[Intel-gfx] ✗ Fi.CI.BAT: failure for series starting with [1/4] dma-buf: add caching of sg_table

[Daniel Vetter]

On Wed, Jul 18, 2018 at 03:24:26PM +0200, Christian König wrote:
> Hi Daniel,
> 
> Am 18.07.2018 um 14:07 schrieb Patchwork:
> > == Series Details ==
> > 
> > Series: series starting with [1/4] dma-buf: add caching of sg_table
> > URL   : https://patchwork.freedesktop.org/series/46778/
> > State : failure
> > [SNIP]
> 
> it looks like I'm a step further understanding the problems which come with
> this change.
> 
> I've more or less audited all use cases and think that only i915 is left
> with the following lock inversion: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_9705/fi-cfl-8700k/igt@gem_mmap_gtt@basic-small-bo-tiledx.html
> 
> Now my question is what is &obj->mm.lock used for and why do you guys call
> dma_buf_map_attachment() while holding it?

obj->mm.lock is the lock to protect getting at the backing storage.
i915_gem_object_get_pages and _put_pages are the relevant functions.

Existing paths want to pin the backing storage while holding the
reservation lock. And your new path needs to do the inverse since
dma_buf_map_attachment now also requires the reservation lock. And that is
obviously called from within the dma-buf importer version of get_pages.

I think there's 2 solutions:

- Merge obj->mm.lock and the reservation lock. Probably the cleaner
  solution, but likely more work.

- Make sure the obj->mm.lock always nests within the reservation lock, and
  grab the reservation lock anywhere it's not yet grabbed. Then you can
  use the dma_buf_map_attachment_locked variant in
  i915_gem_object_get_pages_dmabuf to avoid the locking inversion. This
  would essentially make the obj->mm.lock fully redundant.

Either way is going to be quite a bit of work. I expect that you need to
replace all the cases of dma_buf_map_attachment in i915 with
dma_buf_map_attachment_locked, and adjust the entire callchain to the new
locking scheme.

The real trouble here imo is that i915 CI is just the canary, I expect a
bunch of other drivers will also look at an inverted locking hierarchy if
dma_buf_map_attachment needs the reservation lock. And there's no
convenient CI for them, and code audit won't cut it really (at least I'm
too stupid to keep the locking hierarchy of an entire driver in my head).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch