some half-baked ttm ideas

[Christian König]

Am 16.09.20 um 08:44 schrieb Thomas Hellström (Intel):
>
> On 9/16/20 6:28 AM, Dave Airlie wrote:
>> On Wed, 16 Sep 2020 at 14:19, Dave Airlie <airlied at gmail.com> wrote:
>>> On Wed, 16 Sep 2020 at 00:12, Christian König
>>> <ckoenig.leichtzumerken at gmail.com> wrote:
>>>> Hi Dave,
>>>>
>>>> I think we should just completely nuke ttm_tt_bind() and 
>>>> ttm_tt_unbind()
>>>> and all of that.
>>>>
>>>> Drivers can to this from their move_notify() callback now instead.
>>> Good plan, I've put a bunch of rework into the same branch,
>>>
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fairlied%2Flinux%2Fcommits%2Fttm-half-baked-ideas&data=02%7C01%7Cchristian.koenig%40amd.com%7C3d6c9abffa5543a3881008d85a0c013c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637358354912162169&sdata=kaR1uC3yLF%2F8AT1bgTpo%2F8An%2B0yywHZBAfXTD%2Ft%2BMb0%3D&reserved=0 
>>>
>>>
>>> but I've fried my brain a bit, I'm having trouble reconciling move
>>> notify and unbinding in the right places, I feel like I'm circling
>>> around the answer but haven't hit it yet.
>> drm/ttm: add unbind to move notify paths.
>>
>> In that tree is incorrect and I think where things fall apart, since
>> if we are moving TTM to VRAM that will unbind the TTM object from the
>> GTT at move notify time before the move has executed.
>>
>> I'm feeling a move_complete_notify might be an idea, but I'm wondering
>> if it's a bad idea.
>>
>> Dave.
>
> I don't know if this complicates things more, but move_notify was 
> originally only thought to be an invalidation callback, and was never 
> intended to drive any other actions in the driver than to invalidate 
> various GPU bindings.

And exactly that's what we need to change. See TTM or more precisely the 
eviction handling should only manage where buffers are located and 
notify that driver that something needs to move.

Managing the whole binding/unbinding and actually moving the buffer 
around inside TTM was a bad idea to begin with.

In other words we have domains A, B, C.... and manage which BOs are 
inside those domains and can be evicted when necessary.

If the need arise to evict something the driver gets a notification 
which BO was picked for eviction and acts accordingly.

This means that the eviction_valuable, evict_flag, move_notify, binding, 
unbinding etc... callbacks should be removed in the long term.

Regards,
Christian.

>
> The idea was that TTM should really never set up any GPU bindings, but 
> just provide memory where it was gpu-bindable and make sure it was 
> CPU-mappable where needed. The "exception" was mappable AGP-type 
> gpu-bindings, for the simple reason that they were needed to provide 
> CPU-mappings on systems where you couldn't map the pages directly. But 
> since we set up a GPU map on these systems anyway, many (most) drivers 
> just made use of that, but others took the step further insisting on 
> using move_notify() to set up GPU bindings, which was never intended 
> and adds error paths in the TTM move code that are pretty hard to follow.
>
> So if we're changing things here,  I'd vote for the following:
>
> * Driver calls ttm_bo_validate to put memory where it is cpu-mappable 
> and gpu-bindable
> * On successful validate, driver sets up GPU bindings itself.
>
> * move_notify only invalidates GPU bindings and should really return a 
> void.
>
> So that bind() and unbind() stuff is really only needed for cpu-map 
> through aperture. If we ditch that, then we need to re-define the task 
> of TTM to provide memory in a cpu-mappable location and figure how 
> drivers that require cpu-map-through-aperture should handle this, 
> since they can't use the TTM fault handler for that memory anymore. 
> The same holds for drivers that want to manage their translation table 
> themselves, and needs some cpu-mapping operations to go through the 
> aperture rather than to the pages directly.
>
> If the driver has no special cpu-mapping requirements, it should be 
> perfectly legal for it to not provide any bind() or unbind() 
> functionality.
>
> /Thomas
>
>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel at lists.freedesktop.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&data=02%7C01%7Cchristian.koenig%40amd.com%7C3d6c9abffa5543a3881008d85a0c013c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637358354912162169&sdata=NpShrIj0SED1R8V4JYDAUas7m5jEBYM0MGjSxnU%2FDFc%3D&reserved=0 
>>