[Beignet] [Intel-gfx] Preventing zero GPU virtual address allocation

[Zhigang Gong]

On Mon, Mar 16, 2015 at 01:10:28PM -0700, Jesse Barnes wrote:
> On 03/16/2015 01:52 AM, Daniel Vetter wrote:
> > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Daniel Vetter [mailto:daniel.vetter at ffwll.ch] On Behalf Of Daniel
> >>> Vetter
> >>> Sent: Saturday, March 14, 2015 1:14 AM
> >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling;
> >>> Vetter, Daniel; intel-gfx at lists.freedesktop.org; Yang, Rong R;
> >>> beignet at lists.freedesktop.org
> >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address
> >>> allocation
> >>>
> >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote:
> >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote:
> >>>>> If supporting systems without full ppgtt is a requirement for you
> >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo
> >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That
> >>>>> one will work everywhere. softpin can unexpectedly fail without full
> >>>>> ppgtt if the kernel decides to put something at a given spot, which
> >>>>> imo means we should only expose it on full ppgtt systems.
> >>>>>
> >>>>> And PIN_BIAS should be fairly easy to wire up since the internal
> >>>>> logic is all there already. So "just" needs an execbuf flag, igt
> >>>>> test and appropriate userspace to set that new bit.
> >>>>
> >>>> It doesn't though. To provide the guarantee userspace is asking for
> >>>> (which is that address 0 goes to a special, preferrably inaccessible,
> >>>> page), you have to evict the first N pages in the GGTT. That is just
> >>>> as likely to fail with an execbuffer flag as it would with an execobject flag.
> >>>
> >>> Afaiui userspace only needs the guarantee that NULL is never a valid address.
> >>> Which means it's never a valid address for its own buffer objects. I don't
> >>> think it cares one bit what's actually there, it's not mandatory to fault
> >>> apparently. And faulting is what's not possible.
> >> Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language.
> >> But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension.
> >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation.
> >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address?
> >> So that PIN_BIAS is also effective on the shared buffer, right?
> > 
> > Yeah we'll rebind if needed. We can make this an execbuf or context flag,
> > in either case anything that gets executed by ocl will be moved around if
> > it accidentally ended up at the wrong place. The only exception is if a
> > buffer is pinned already, i.e. if you're doing direct rendering to the
> > frontbuffer. That will give you an EBUSY, but otoh that also shouldn't
> > ever happen really.
> 
> Ruiling, are you working on this or someone from your team, presumably
> based on the patch Chris posted earlier?  The zero page reservation
> certainly seems simpler to me, but the MAP_FIXED approach is a lot more
> flexible, and can be used for other types of debug and usages as well
> (we'll need something like it for OCL pointer sharing for example), so
> seems like a good thing to pursue regardless.

Jesse, here is my 2 cent comment:

IMO, the start of this discussion is from a very simple issue:
1. Make sure any valid buffer object will not be mapped to zero page.
   OCL wants to avoid 0 virtual address for any valid buffer object. Thus
   the following type of code will work as expected.
   __kernel foo(__global int *src, __global int* dst) {
     if (dst == NULL)
       return;
   }
   Although the above type code may not make much sense, but it should
   be supported as well.
   This is a simple requirement for all the bo's binding operations from
   CL application. And we think that simply skip the zero-page in KMD is
   a good enough solution and don't see any side-effect till now.

Then let's talk about the MAP_FIXED approach:
2. About the MAP_FIXED approach.
  If my understanding is correct, this is a intermediate solution towards FULL
  SVM function.
  From OCL's view, it is usefull for the 2.0's. There are three types of SVM
  options. Even for the most basic "Coarse-Grained buffer SVM", something like
  MAP_FIXED is useful. Becuase it needs to pass a linked list object to the
  OCL kernel directly. If we can use MAP_FIXED to map the bo to the VM address
  which is used by host, then we can access such a linked list easily on OCL
  kernel side, otherwise, we may need some tricks to do extra address adjustment.
  So this feature is useful for Beignet before we get full SVM support.

My conclusion is:
* The 1st is a totally *passive* requirement and need to be applied to each
  buffer binding. And it's better to be transparent to userspace.
* The 2nd is an *active* requirement and need the kernel to provide SVM like
  interface to the user space. And only those SVM buffers (for the first 2 OCL
  SVM options) need to call the new interfaces explicitly.

Thanks,
Zhigang Gong.

> 
> Thanks,
> Jesse
> 
> _______________________________________________
> Beignet mailing list
> Beignet at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/beignet