Possible use_mm() mis-uses

[Christian König]

Adding Felix because the KFD part of amdgpu is actually his responsibility.

If I'm not completely mistaken the release callback of the mmu_notifier 
should take care of that for amdgpu.

Regards,
Christian.

Am 22.08.2018 um 18:44 schrieb Linus Torvalds:
> Guys and gals,
>   this is a *very* random list of people on the recipients list, but we
> had a subtle TLB shootdown issue in the VM, and that brought up some
> issues when people then went through the code more carefully.
>
> I think we have a handle on the TLB shootdown bug itself. But when
> people were discussing all the possible situations, one thing that
> came up was "use_mm()" that takes a mm, and makes it temporarily the
> mm for a kernel thread (until "unuse_mm()", duh).
>
> And it turns out that some of those uses are definitely wrong, some of
> them are right, and some of them are suspect or at least so overly
> complicated that it's hard for the VM people to know if they are ok.
>
> Basically, the rule for "use_mm()" is that the mm in question *has* to
> have a valid page table associated with it over the whole use_mm() ->
> unuse_mm() sequence. That may sound obvious, and I guess it actually
> is so obvious that there isn't even a comment about it, but the actual
> users are showing that it's sadly apparently not so obvious after all.
>
> There is one user that uses the "obviously correct" model: the vhost
> driver does a "mmget()" and "mmput()" pair around its use of it,
> thanks to vhost_dev_set_owner() doing a
>
>          dev->mm = get_task_mm(current);
>
> to look up the mm, and then the teardown case does a
>
>          if (dev->mm)
>                  mmput(dev->mm);
>          dev->mm = NULL;
>
> This is the *right* sequence. A gold star to the vhost people.
>
> Sadly, the vhost people are the only ones who seem to get things
> unquestionably right. And some of those gold star people are also
> apparently involved in the cases that didn't get things right.
>
> An example of something that *isn't* right, is the i915 kvm interface,
> which does
>
>          use_mm(kvm->mm);
>
> on an mm that was initialized in virt/kvm/kvm_main.c using
>
>          mmgrab(current->mm);
>          kvm->mm = current->mm;
>
> which is *not* right. Using "mmgrab()" does indeed guarantee the
> lifetime of the 'struct mm_struct' itself, but it does *not* guarantee
> the lifetime of the page tables. You need to use "mmget()" and
> "mmput()", which get the reference to the actual process address
> space!
>
> Now, it is *possible* that the kvm use is correct too, because kvm
> does register a mmu_notifier chain, and in theory you can avoid the
> proper refcounting by just making sure the mmu "release" notifier
> kills any existing uses, but I don't really see kvm doing that. Kvm
> does register a release notifier, but that just flushes the shadow
> page tables, it doesn't kill any use_mm() use by some i915 use case.
>
> So while the vhost use looks right, the kvm/i915 use looks definitely wrong.
>
> The other users of "use_mm()" and "unuse_mm()" are less
> black-and-white right vs wrong..
>
> One of the complex ones is the amdgpu driver. It does a
> "use_mm(mmptr)" deep deep in the guts of a macro that ends up being
> used in fa few places, and it's very hard to tell if it's right.
>
> It looks almost certainly buggy (there is no mmget/mmput to get the
> refcount), but there _is_ a "release" mmu_notifier function and that
> one - unlike the kvm case - looks like it might actually be trying to
> flush the existing pending users of that mm.
>
> But on the whole, I'm suspicious of the amdgpu use. It smells. Jann
> Horn pointed out that even if it migth be ok due to the mmu_notifier,
> the comments are garbage:
>
>>   Where "process" in the uniquely-named "struct queue" is a "struct
>>   kfd_process"; that struct's definition has this comment in it:
>>
>>         /*
>>          * Opaque pointer to mm_struct. We don't hold a reference to
>>          * it so it should never be dereferenced from here. This is
>>          * only used for looking up processes by their mm.
>>          */
>>         void *mm;
>>
>>   So I think either that comment is wrong, or their code is wrong?
> so I'm chalking the amdgpu use up in the "broken" column.
>
> It's also actually quite hard to synchronze with some other kernel
> worker thread correctly, so just on general principles, if you use
> "use_mm()" it really really should be on something that you've
> properly gotten a mm refcount on with mmget(). Really. Even if you try
> to synchronize things.
>
> The two final cases are two uses in the USB gadget driver. Again, they
> don't have the proper mmget/mmput, but they may br ok simply because
> the uses are done for AIO, and the VM teardown is preceded by an AIO
> teardown, so the proper serialization may come in from that.
>
> Anyway, sorry for the long email, and the big list of participants and
> odd mailing lists, but I'd really like people to look at their
> "use_mm()" cases, and ask themselves if they have done enough to
> guarantee that the full mm exists. Again, "mmgrab()" is *not* enough
> on its own. You need either "mmget()" or some lifetime guarantee.
>
> And if you do have those lifetime guarantees, it would be really nice
> to get a good explanatory comment about said lifetime guarantees above
> the "use_mm()" call. Ok?
>
> Note that the lifetime rules are very important, because obviously
> use_mm() itself is never called in the context of the process that has
> the mm. By definition, we're in a kernel thread and it uses somebody
> elses mm. So it's important to show that that "somebody else" really
> is serialized with the kernel thread somehow, and keeps the mm alive.
> The easiest way by far to have that guarantee is to have that
> "mmget/mmput" model.
>
> Please? Even if you're convinced your code is right, just a comment
> about _why_ it's right, ok?
>
> And no, use_mm() cannot just do the mmget internally, only to have
> unuse_mm() do the mmput().  That was our first reaction when looking
> at this, but if the caller has screwed up the lifetime rules, it's not
> actually clear that the mm is guaranteed to be around even when use_mm
> is entered. So the onus of proper lifetime rules really is on the
> caller, not on use_mm()/unuse_mm().
>
>               Linus