[Intel-gfx] [PATCH 20/41] drm/i915: Replace priolist rbtree with a skiplist

[Matthew Brost]

On Fri, Jan 29, 2021 at 10:30:58AM +0000, Chris Wilson wrote:
> Quoting Matthew Brost (2021-01-28 22:56:04)
> > On Mon, Jan 25, 2021 at 02:01:15PM +0000, Chris Wilson wrote:
> > > Replace the priolist rbtree with a skiplist. The crucial difference is
> > > that walking and removing the first element of a skiplist is O(1), but
> > > O(lgN) for an rbtree, as we need to rebalance on remove. This is a
> > > hindrance for submission latency as it occurs between picking a request
> > > for the priolist and submitting it to hardware, as well effectively
> > > trippling the number of O(lgN) operations required under the irqoff lock.
> > > This is critical to reducing the latency jitter with multiple clients.
> > > 
> > > The downsides to skiplists are that lookup/insertion is only
> > > probablistically O(lgN) and there is a significant memory penalty to
> > > as each skip node is larger than the rbtree equivalent. Furthermore, we
> > > don't use dynamic arrays for the skiplist, so the allocation is fixed,
> > > and imposes an upper bound on the scalability wrt to the number of
> > > inflight requests.
> > > 
> > 
> > This is a fun data structure but IMO might be overkill to maintain this
> > code in the i915. The UMDs have effectively agreed to use only 3 levels,
> > is O(lgN) where N == 3 really a big deal? With GuC submission we will
> > statically map all user levels into 3 buckets. If we are doing that, do
> > we even need a complex data structure? i.e. Could use just use can
> > array of linked lists?
> 
> Because we need to scale the bst to handle a unqiue key per request with
> thousands of requests [this is not only about priorities]. And as you
> will see from the results, even with just a single priority in the system
> (so one entry in either the skiplist or rbtree), the skiplist is beating 
> the rbtree as measured by the lock hold time around insert/dequeue of
> requests. That surprised me.

Ok, seems reasonable. Skips list are pretty cool, wondering if at some
point we should make skip list code a bit more generic so it can
possibly be levered in other parts of the i915 / kernel.

Matt

> -Chris