[Intel-gfx] [RFC 53/60] drm/i915: Create stolen memory region from local memory

[Tang, CQ]

> -----Original Message-----
> From: Joonas Lahtinen <joonas.lahtinen at linux.intel.com>
> Sent: Friday, August 7, 2020 2:39 AM
> To: Tang, CQ <cq.tang at intel.com>; Dave Airlie <airlied at gmail.com>
> Cc: Auld, Matthew <matthew.auld at intel.com>; Intel Graphics Development
> <intel-gfx at lists.freedesktop.org>; Abdiel Janulgue
> <abdiel.janulgue at linux.intel.com>; Wilson, Chris P
> <chris.p.wilson at intel.com>; Balestrieri, Francesco
> <francesco.balestrieri at intel.com>; Vishwanathapura, Niranjana
> <niranjana.vishwanathapura at intel.com>; Dhanalakota, Venkata S
> <venkata.s.dhanalakota at intel.com>; Neel Desai <neel.desai at intel.com>;
> Brost, Matthew <matthew.brost at intel.com>; Dutt, Sudeep
> <sudeep.dutt at intel.com>; De Marchi, Lucas <lucas.demarchi at intel.com>
> Subject: Re: [RFC 53/60] drm/i915: Create stolen memory region from local
> memory
> 
> Quoting Dave Airlie (2020-07-14 22:26:16)
> > On Wed, 15 Jul 2020 at 02:57, Tang, CQ <cq.tang at intel.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Auld, Matthew <matthew.auld at intel.com>
> > > > Sent: Tuesday, July 14, 2020 8:02 AM
> > > > To: Dave Airlie <airlied at gmail.com>
> > > > Cc: Intel Graphics Development <intel-gfx at lists.freedesktop.org>;
> > > > Tang, CQ <cq.tang at intel.com>; Joonas Lahtinen
> > > > <joonas.lahtinen at linux.intel.com>;
> > > > Abdiel Janulgue <abdiel.janulgue at linux.intel.com>; Wilson, Chris P
> > > > <chris.p.wilson at intel.com>; Balestrieri, Francesco
> > > > <francesco.balestrieri at intel.com>; Vishwanathapura, Niranjana
> > > > <niranjana.vishwanathapura at intel.com>; Dhanalakota, Venkata S
> > > > <venkata.s.dhanalakota at intel.com>; Neel Desai
> > > > <neel.desai at intel.com>; Brost, Matthew <matthew.brost at intel.com>;
> > > > Dutt, Sudeep <sudeep.dutt at intel.com>; De Marchi, Lucas
> > > > <lucas.demarchi at intel.com>
> > > > Subject: Re: [RFC 53/60] drm/i915: Create stolen memory region
> > > > from local memory
> > > >
> > > > On 13/07/2020 05:48, Dave Airlie wrote:
> > > > > On Fri, 10 Jul 2020 at 22:01, Matthew Auld
> > > > > <matthew.auld at intel.com>
> > > > wrote:
> > > > >>
> > > > >> From: CQ Tang <cq.tang at intel.com>
> > > > >>
> > > > >> Add "REGION_STOLEN" device info to dg1, create stolen memory
> > > > >> region from upper portion of local device memory, starting from
> DSMBASE.
> > > > >>
> > > > >> The memory region is marked with "is_devmem=true".
> > > > >
> > > > > So is stolen fake on LMEM devices? The concept of stolen doesn't
> > > > > seem to make much sense with VRAM, so please enlighten me.
> > > >
> > > > CQ, do we actually need an explicit stolen LMEM region? The idea
> > > > of having a DSM like stolen region for LMEM does sound
> > > > strange(outside of the usual reserved portions which are for HW
> > > > use etc), since the driver has complete control over LMEM. Is it
> > > > just a convenience thing to keep things working as-is for fbc, initial fb,
> etc. or is there more to it?
> > > > There is buddy_alloc_range() for LMEM which we could potentially
> > > > use to wrap an object around for things like the initial fb or similar.
> > >
> > > This is a natural extension from IGT stolen memory region into DGT, we
> want to allocate objects from stolen area. In DGT, we have one stolen area
> per tile so we create one region in each of these area. Using memory region
> is easier to manage objects allocation and free. Other than fbc and rc6, we
> have gt/ring allocate stolen memory objects when without LMEM, so only
> apply to IGT case:
> > >
> > > display/intel_display.c:        obj =
> i915_gem_object_create_stolen_for_preallocated(dev_priv,
> > > display/intel_fbdev.c:                  obj =
> i915_gem_object_create_stolen(dev_priv, size);
> > > display/intel_overlay.c:        obj = i915_gem_object_create_stolen(i915,
> PAGE_SIZE);
> > > intel_rc6.c:            pctx =
> i915_gem_object_create_stolen_for_preallocated(i915,
> > > intel_rc6.c:    pctx = i915_gem_object_create_stolen(i915, pctx_size);
> > >
> > > intel_ring.c:                   obj = intel_gt_object_create_stolen(ggtt->vm.gt,
> size);
> > > intel_gt.c:             obj = intel_gt_object_create_stolen(gt, size);
> > >
> > > For some reason, we don't use buddy allocator to manage the stolen
> memory, instead, we use drm_mm_node allocator directly, we have one-to-
> one mapping between drm_mm address space to dma address of the stolen
> memory. We also use contiguous allocation where an object always get a
> single contiguous block of pages.
> > >
> > > So fundamentally, we want to use the same code to work on both IGT
> stolen memory and DGT stolen memory.
> >
> > If this is fundamentally a software construct then it's horrible, if
> > the HW has a stolen base like Ville said, and it needs to be in a
> > chunk of VRAM, how do you go about sizing that, and carving it out
> > from the user?
> >
> > I don't think wanting to share the same codepaths here is an
> > acceptable answer, just fix the code to handle LMEM vs stolen, but if
> > there are hw reasons on why this is required it would be good to
> > enumerate exactly what they are and document them.
> 
> The boot firmware does configure the stolen region, so it's not fake.
> 
> But there's no need to expose the stolen memory to userspace, so these
> patches will be dropped.

Yes, I confirmed that on discrete GPU with display, firmware does reserve a portion of device local memory as stolen memory region, and used by device hardware, the usage of this local stolen memory is the same as system stolen memory on integrated GPU.

So we need to continue to create this local stolen memory region in driver. However, because this stolen memory region is only used by driver and hardware, there is no need to expose the stolen memory region to userspace.

--CQ

> 
> Regards, Joonas