[Intel-gfx] [PATCH] drm/i915/gt: Defend against concurrent updates to execlists->active

[Mika Kuoppala]

Chris Wilson <chris at chris-wilson.co.uk> writes:

> Quoting Mika Kuoppala (2020-03-09 15:34:40)
>> Chris Wilson <chris at chris-wilson.co.uk> writes:
>> 
>> > [  206.875637] BUG: KCSAN: data-race in __i915_schedule+0x7fc/0x930 [i915]
>> > [  206.875654]
>> > [  206.875666] race at unknown origin, with read to 0xffff8881f7644480 of 8 bytes by task 703 on cpu 3:
>> > [  206.875901]  __i915_schedule+0x7fc/0x930 [i915]
>> > [  206.876130]  __bump_priority+0x63/0x80 [i915]
>> > [  206.876361]  __i915_sched_node_add_dependency+0x258/0x300 [i915]
>> > [  206.876593]  i915_sched_node_add_dependency+0x50/0xa0 [i915]
>> > [  206.876824]  i915_request_await_dma_fence+0x1da/0x530 [i915]
>> > [  206.877057]  i915_request_await_object+0x2fe/0x470 [i915]
>> > [  206.877287]  i915_gem_do_execbuffer+0x45dc/0x4c20 [i915]
>> > [  206.877517]  i915_gem_execbuffer2_ioctl+0x2c3/0x580 [i915]
>> > [  206.877535]  drm_ioctl_kernel+0xe4/0x120
>> > [  206.877549]  drm_ioctl+0x297/0x4c7
>> > [  206.877563]  ksys_ioctl+0x89/0xb0
>> > [  206.877577]  __x64_sys_ioctl+0x42/0x60
>> > [  206.877591]  do_syscall_64+0x6e/0x2c0
>> > [  206.877606]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>> >
>> > References: https://gitlab.freedesktop.org/drm/intel/issues/1318
>> > Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
>> > ---
>> >  drivers/gpu/drm/i915/gt/intel_engine.h | 12 +++++++++++-
>> >  1 file changed, 11 insertions(+), 1 deletion(-)
>> >
>> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
>> > index 29c8c03c5caa..f267f51c457c 100644
>> > --- a/drivers/gpu/drm/i915/gt/intel_engine.h
>> > +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
>> > @@ -107,7 +107,17 @@ execlists_num_ports(const struct intel_engine_execlists * const execlists)
>> >  static inline struct i915_request *
>> >  execlists_active(const struct intel_engine_execlists *execlists)
>> >  {
>> > -     return *READ_ONCE(execlists->active);
>> > +     struct i915_request * const *cur = READ_ONCE(execlists->active);
>> > +     struct i915_request * const *old;
>> > +     struct i915_request *active;
>> > +
>> > +     do {
>> > +             old = cur;
>> > +             active = READ_ONCE(*cur);
>> > +             cur = READ_ONCE(execlists->active);
>> > +     } while (cur != old);
>> > +
>> > +     return active;
>> 
>> The updated side is scary. We are updating the execlists->active
>> in two phases and handling the array copying in between.
>> 
>> as WRITE_ONCE only guarantees ordering inside one context, due to
>> it is for compiler only, it makes me very suspicious about
>> how the memcpy of pending->inflight might unravel between two cpus.
>> 
>> smb_store_mb(execlists->active, execlists->pending);
>> memcpy(inflight, pending)
>> smb_wmb();
>> smb_store_mb(execlists->active, execlists->inflight);
>> smb_store_mb(execlists->pending[0], NULL);
>
> Not quite. You've overkill on the mb there.
>
> If you want to be pedantic,
>
> WRITE_ONCE(active, pending);
> smp_wmb();
>
> memcpy(inflight, pending);
> smp_wmb();
> WRITE_ONCE(active, inflight);

This is the crux of it, needing rmb counterpart.
-Mika

>
> The update of pending is not part of this sequence.
>
> But do we need that, and I still think we do not.
>
>> This in paired with:
>> 
>> active = READ_ONCE(*cur);
>> smb_rmb();
>> cur = READ_ONCE(execlists->active);
>> 
>> With this, it should not matter at which point the execlists->active
>> is sampled as the pending would be guaranteed to be
>> immutable if it sampled early and inflight immutable if it
>> sampled late?
>
> Simply because we don't care about the sampling, just that the read
> dependency gives us a valid pointer. (We are not looking at a snapshot
> of several reads, but a _single_ read and the data dependency from
> that.)
> -Chris