TTM's role in score-based eviction

[Thomas Hellstrom]

Hi!

On 12/05/2013 10:36 AM, Lauri Kasanen wrote:
> Hi list, Thomas,
>
> I will be investigating the use of a hotness score for each bo, to
> replace the ping-pong causing LRU eviction in radeon*.
>
> The goal is to put all bos that fit in VRAM there, in order of hotness;
> a new bo should only be placed there if its hotness score is greater
> than the lowest VRAM bo's. Then the lowest-hotness-bos in
> VRAM should be evicted until the new bo fits. This should result in a
> more stable set with less ping-pong.
>
> Jerome advised that the bo placement should be done entirely outside
> TTM. As I'm not (yet) too familiar with that side of the kernel, what is
> the opinion of TTM folks?

There are a couple of things to be considered:
1) You need to decide where a bo to be validated should be placed. The 
driver can give a list of possible placements to TTM and let
TTM decide, trying each placement in turn. A driver that thinks this 
isn't sufficient can come up with its on strategy and give only a single 
placement to TTM. If TTM can't satisfy that, it will give you an error 
back, and the driver will need to validate with an alternative 
placement. I think Radeon already does this? vmwgfx does it to some extent.

2) As you say, TTM is evicting strictly on an lru basis, and is 
maintaining one LRU list per memory type, and also a global swap lru 
list for buffers that are backed by system pages (not VRAM). I guess 
what you would want to do is to replace the VRAM lru list with a 
priority queue where bos are continously sorted based on hotness.
As long as you obey the locking rules:
*) Locking order is bo::reserve -> lru-lock
*) When walking the queue with the lru-lock held, you must therefore 
tryreserve if you want to reserve an object on the queue
*) bo:s need to be removed from the queue as soon as they are reserved
*) Don't remove a bo from the queue unless it is reserved
Nothing stops you from doing this in the driver, but OTOH if this ends 
up being useful for other drivers I'd prefer we put it into TTM.

Thanks,
Thomas



>
> - Lauri
>
> * github.com/clbr/jamkthesis