[PATCH split 3/3] Add support for high priority scheduling in amdgpu v9

[Andres Rodriguez]

Third part of the split of the series:
Add support for high priority scheduling in amdgpu v8

This is the part of the series that is in a bit more murky water than the rest.

Sending out this patch series mostly for completion. And maybe for discussion
purposes as well. There are still 2 issues open with this series:

  1) Is the spinlock patch still okay? Should we pursue this differently?

I'd rather not use a mutex here. That would mean that to program srbm registers
from an interrupt we'd need to dispatch a worker thread. That could mean extra
time that the CU reservation is in place which can impact performance.

So my preferred (biased) alternative is to still move to a spinlock.

Another alternative I'm not sure of: Can we take advantage of the KIQ FIFO
semantics to perform srbm writes atomically?

Something like:
   ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(...)))
   ib_append(ib, PKT_WRITE_REG(SOME_REG, VAL)
   ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(0, 0, 0)))
   ib_sumbit(kiq_ring, ib)

Something that makes this immediately feel wrong though is the possibility of a
race condition between an srbm operation on the KIQ and one through MMIO.

  2) Alex suggested changing some MMIO writes to happen on the KIQ instead. I
still haven't addressed that.

I'm not sure the full criteria for patches landing on -wip. But if these are good enough
to fix with some followup work, I wouldn't be oppossed to that idea.