Where do ttm->pages come from?

[Christian König]

Am 18.03.25 um 16:01 schrieb Matthew Wilcox:
> On Tue, Mar 18, 2025 at 09:37:42AM +0100, Christian König wrote:
>> Hi Matthew,
>>
>> you actually got your understanding exactly inverted :)
> Thanks; that's very helpful!
>
>> Amdgpu takes a look at page->mapping to make sure they are *not* coming from the call sequence starting with amdgpu_ttm_tt_set_user_pages() or be imported from another driver through DMA-buf etc...
>>
>> The only pages which are allowed to pass this check are the private driver allocated ones, see this code function amdgpu_ttm_tt_populate():
>>
>>         for (i = 0; i < ttm->num_pages; ++i)
>>                 ttm->pages[i]->mapping = bdev->dev_mapping;
>>
>> So far that has been save since nobody else should be using the address space object from our drivers inode under /dev.
>>
>> But when you plan to remove page->mapping we probably have a problem here.
> OK.  This makes the problem a lot more tractable.
>
> From the MM side, we have a short-term and a long-term goal.  The
> long-term goal is to reduce struct page to a single tagged pointer
> called a memdesc.  Details here:
>
> https://kernelnewbies.org/MatthewWilcox/Memdescs
>
> In the short term, we're removing page->mapping, ->index, ->lru and
> ->memcg_data, leaving ->flags, ->_refcount and ->private as well as the
> newly-introduced ->memdesc.
>
> So if all you needed to do was identify which pages belong to you,
> I'd suggest setting ->private to a unique value (eg bdev->dev_mapping).

The problem is ->private is already used for something else in that use case. See the code in ttm_pool.c.

> But it feels like you need more than that; you need a refcount, and you
> need to map pages to userspace.  Do you need to support GUP on these
> pages (ie can it be the user buffer for O_DIRECT, can it be registered
> with RDMA, that kind of thing), or is this memory just for userspace to
> do load/store, and no fancy stuff is allowed?

GUP is actually strictly forbidden. We have a repeating problem with people trying to enable that and running into data corruption after a while.

Userspace mapping is just using the PFN, so no refcount needed either.

> At this point, I'd suggest that you allocate folios rather than pages.
> That lets us continue to use folio->mapping to distinguish these pages.
> There would be an opportunity to save some memory in the future by using
> a slimmer control structure once we understand the new system better,
> but maybe it's OK to just keep using a folio forever.
>
> Thoughts?

Yeah, switching to folios in TTM is on the TODO list for like as long as folios exist. But that is easier said than done.

Regards,
Christian.