[Intel-gfx] [RFC 3/3] drm/i915: Add the truncation logic for Stolen objects.

[Gupta, Sourab]

Hi Daniel,

We had also considered the two points where the stolen mem objs can be truncated -
	a) at the point of a new bo allocation, if stolen mem space is exhausted. 
	b) at the point when the bo is marked as purgeable (madvise ioctl)

We decided upon the second case because:
1) For normal objects, there is a bo cacheing at libdrm level. So even after a bo is marked as purgeable, 
it can still get reused by libdrm (if not truncated already), thereby reducing the effort in new object creation. 
Currently for stolen objects we have not enabled this cacheing. So once a stolen bo is marked as purgeable by User, 
Driver can immediately reclaim the stolen space and there will be no obvious benefit in deferring the object truncation to 
a later stage (when there is a failure in allocation of a new bo). 
Actually we thought that since Stolen mem is a relatively much smaller space and has a limited usage,  
there may not be any additional benefit gained by enabling bo cacheing on libdrm side. 
Also the effort in getting the backing space from stolen mem shall be much less than from shmem. 

With the first approach, it would have required us to scan the entire list of bound & unbound objects to search for the 
purgeable stolen objects and purge them. With the 2nd approach, we could avoid this & purge the objects we already have
in our hand. 

2) Also, there is no coupling between the linux shrinker & the stolen memory. Shrinker invocation will depend upon the  
system memory usage. So, we cannot rely on shrinker to timely trigger the purging of objects from stolen mem.

We understand that we are diverging from the basic nature of madvise ioctl here (which is to only mark objs
as purgeable and so eligible for truncation if needed), but probably because of the above way of using the stolen memory, 
we thought madvise could also be one of the good checkpoint where we can truncate the stolen bos.

What are your views regarding above.
We will try to revisit our design if its digressing too much from the basic tenets regarding the madvise ioctl 

Thanks,
Sourab


-----Original Message-----
From: Vetter, Daniel 
Sent: Thursday, March 06, 2014 1:18 AM
To: Gupta, Sourab; Chris Wilson
Cc: intel-gfx at lists.freedesktop.org; Barnes, Jesse; Wilson, Chris; Goel, Akash
Subject: Re: [RFC 3/3] drm/i915: Add the truncation logic for Stolen objects.

On 05/03/2014 20:24, Gupta, Sourab wrote:
> We have assumed the following lifecycle of the (stolen)object, w.r.t the truncation usecase:
>
> 1) The user creates the (non-cpu mappable)object --> the gem object is 
> created. Shmem filep is allocated. No backing storage present
> 2) GPU operations performed (after mmap_gtt ) --> object is moved to stolen memory and the backing pages are allocated from stolen mem area.
> 3) Object is marked as purgeable (by madvise_ioctl)-->  here two scenarios can occur:
> 	a) Object has not yet been processed by GPU. In this case it will be in the active list. So, madvise_ioctl will mark it as purgeable and return.
> 	     Subsequently, when the retire handler runs, and object is being moved to inactive list, then, the purgeable object is truncated and marked as purged.
> 	b) Object is already processed by the GPU. In this case, it will be in the inactive list. So, madvise_ioctl will do the object truncation and mark it as purged.
> 4) If object is not marked as purgeable, the backing pages of the object will remain.
> 5) When object is freed, backing pages are released, if the object has not been purged already.
>
> Is the object being in the inactive list, the necessary and sufficient condition of it not being in use?
> We may be missing out on some condition regarding when the object will be in use.
> Can you please point us to the flaws in above assumption regarding when we can truncate the object.
Immediately purging the backing storage is not the idea of the madvise ioctl. The idea is to purge it as late as possible, i.e. when we've tried to allocate more stolen space but failed. For comparison see how purgeable works for normal objects: Those only get purged in the shrinker, i.e. when the linux mm is short on memory.
-Daniel