[PATCH 1/2] drm/amdgpu: Enable scatter gather display support

[Alex Deucher]

On Mon, Mar 19, 2018 at 4:26 PM, Marek Olšák <maraeo at gmail.com> wrote:
> When Mesa wants a buffer in VRAM, it always sets VRAM. It relies on BO move
> throttling to prevent unnecessary BO moves.
>
> My questions are:
> - what should Mesa do differently for tiny VRAM?
> - what is a tiny VRAM?
> - if VRAM is tiny, which allocations should we put there?

Right now, all mesa would have to do is query the kernel to see if the
device supports s/g display.  If it does, it can do whatever makes
sense in the long term once we've had more time to benchmark, etc.,
but to start off with, I'd say just make it GTT or VRAM|GTT.
Otherwise we have the kernel second guessing mesa and I'd like to
avoid that.  I want the kernel to respect that userspace asks for as
much as possible.

Alex

>
> Marek
>
> On Mon, Mar 19, 2018 at 4:15 PM, Deucher, Alexander
> <Alexander.Deucher at amd.com> wrote:
>>
>> s/not/now/.  I meant to say, “we have NOW excluded the possibility of ever
>> setting displays anywhere else without a kernel update”.
>>
>>
>>
>> Alex
>>
>>
>>
>> From: amd-gfx [mailto:amd-gfx-bounces at lists.freedesktop.org] On Behalf Of
>> Deucher, Alexander
>> Sent: Monday, March 19, 2018 4:13 PM
>>
>>
>> To: Li, Samuel <Samuel.Li at amd.com>; Koenig, Christian
>> <Christian.Koenig at amd.com>; Marek Olšák <maraeo at gmail.com>
>> Cc: Alex Deucher <alexdeucher at gmail.com>; Michel Dänzer
>> <michel at daenzer.net>; amd-gfx list <amd-gfx at lists.freedesktop.org>
>> Subject: Re: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> I'm not sure what you mean by the 3 scenarios.  Generally userspace
>> selects what domains it wants a buffer to be in, vram, gtt, or both (don't
>> care).  I'd rather not have the kernel second guess the UMDs if we can help
>> it.  I'd rather leave the kernel for cases where we have to force things due
>> to hw bugs, or hw restrictions, etc.  If we force all display buffers to be
>> in gtt in the kernel, we have not excluded the possibility of ever setting
>> displays anywhere else without a kernel update.  E.g., to my earlier point,
>> there may be cases where it is advantageous to put display buffers in vram
>> even if s/g display is supported.  That was the point I was trying to make
>> about user mode selecting the domain (vram of gtt or vram|gtt).  Say you
>> have a board with 2 GB of ram and 1 GB is carved out for "vram".  In that
>> case, it would make sense to put the buffer in vram because otherwise you
>> are wasting a comparatively scarce resource.
>>
>>
>>
>> Alex
>>
>> ________________________________
>>
>> From: Li, Samuel
>> Sent: Monday, March 19, 2018 3:58:52 PM
>> To: Deucher, Alexander; Koenig, Christian; Marek Olšák
>> Cc: Alex Deucher; Michel Dänzer; amd-gfx list
>> Subject: RE: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> Alex,
>>
>>
>>
>> I assume you are talking the three scenarios here, 1)VRAM, 2)GTT,
>> 3)VRAM/GTT.
>>
>> But kernel will need the decision too(amdgpufb). I think it shall be
>> better to do it in kernel, instead of different clients(mesa, ddx, kernel …)
>>
>>
>>
>> Regards,
>>
>> Samuel Li
>>
>>
>>
>> From: Deucher, Alexander
>> Sent: Monday, March 19, 2018 3:54 PM
>> To: Li, Samuel <Samuel.Li at amd.com>; Koenig, Christian
>> <Christian.Koenig at amd.com>; Marek Olšák <maraeo at gmail.com>
>> Cc: Alex Deucher <alexdeucher at gmail.com>; Michel Dänzer
>> <michel at daenzer.net>; amd-gfx list <amd-gfx at lists.freedesktop.org>
>> Subject: Re: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> My personal preference is still to plumb this through to mesa rather than
>> forcing it in the kernel.
>>
>>
>>
>> Alex
>>
>> ________________________________
>>
>> From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org> on behalf of Li,
>> Samuel <Samuel.Li at amd.com>
>> Sent: Monday, March 19, 2018 3:50:34 PM
>> To: Koenig, Christian; Marek Olšák
>> Cc: Alex Deucher; Michel Dänzer; amd-gfx list
>> Subject: RE: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> Christian,
>>
>>
>>
>> You misunderstood Alex’s comments,
>>
>>
>>
>> >Regardless of which scenarios we need to support, I think we also need
>>
>> >to really plumb this through to mesa however since user space is who
>>
>> >ultimately requests the location.  Overriding it in the kernel gets
>>
>> >tricky and can lead to ping-ponging as others have noted.  Better to
>>
>>
>>
>> Here Alex mentioned the scenarios is 1)VRAM, 2)GTT, 3)VRAM/GTT.
>>
>> His concern is this might cause ping-pong, not about preferred domain.
>> Since preferred domain can solve the ping-pong issue, it shall address his
>> concern here.
>>
>>
>>
>> Regards,
>>
>> Samuel Li
>>
>>
>>
>> From: Christian König [mailto:ckoenig.leichtzumerken at gmail.com]
>> Sent: Monday, March 19, 2018 3:45 PM
>> To: Li, Samuel <Samuel.Li at amd.com>; Marek Olšák <maraeo at gmail.com>;
>> Koenig, Christian <Christian.Koenig at amd.com>
>> Cc: Alex Deucher <alexdeucher at gmail.com>; Michel Dänzer
>> <michel at daenzer.net>; amd-gfx list <amd-gfx at lists.freedesktop.org>
>> Subject: Re: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> Quoting Alex:
>>
>> Regardless of which scenarios we need to support, I think we also need
>>
>> to really plumb this through to mesa however since user space is who
>>
>> ultimately requests the location.  Overriding it in the kernel gets
>>
>> tricky and can lead to ping-ponging as others have noted.  Better to
>>
>> have user space know what chips support it or not and request display
>>
>> buffers in GTT or VRAM from the start.
>>
>> And I completely agree with Alex here. So overriding the domain in the
>> kernel is a serious NAK from my side as well.
>>
>> Please implement the necessary bits in Mesa, shouldn't be more than a few
>> lines of code anyway.
>>
>> Regards,
>> Christian.
>>
>> Am 19.03.2018 um 20:42 schrieb Li, Samuel:
>>
>> Agreed.
>>
>>
>>
>> >I think that the consensus with Alex and me is that we should avoid
>> > exactly that.
>> Christian, Alex’s concern is about ping-pong, not about the preferred
>> domain.
>>
>> Regards,
>>
>> Samuel Li
>>
>>
>>
>> From: Marek Olšák [mailto:maraeo at gmail.com]
>> Sent: Monday, March 19, 2018 3:39 PM
>> To: Koenig, Christian <Christian.Koenig at amd.com>
>> Cc: Li, Samuel <Samuel.Li at amd.com>; Michel Dänzer <michel at daenzer.net>;
>> Alex Deucher <alexdeucher at gmail.com>; amd-gfx list
>> <amd-gfx at lists.freedesktop.org>
>> Subject: Re: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>>
>>
>> On Mon, Mar 19, 2018 at 3:27 PM, Christian König
>> <christian.koenig at amd.com> wrote:
>>
>> I think that the consensus with Alex and me is that we should avoid
>> exactly that.
>>
>> Overriding the preferred domain in the kernel is a no-go for that patch
>> set, so please implement the discussed changes in Mesa.
>>
>>
>>
>> I don't see how Mesa can make a smarter decision than the kernel. If you
>> overwrite the preferred domain of the buffer in the kernel, there will be no
>> ping-ponging between domains. Mesa never changes the initial preferred
>> domain.
>>
>>
>>
>> Marek
>>
>>
>>
>>
>>
>>
>> Regards,
>> Christian.
>>
>>
>>
>> Am 19.03.2018 um 20:22 schrieb Li, Samuel:
>>
>> I agree with Marek/Michel: since kernel sets the domain before scanning
>> out, it shall update the preferred domain here too.
>>
>> Regards,
>> Samuel Li
>>
>> -----Original Message-----
>> From: Koenig, Christian
>> Sent: Thursday, March 08, 2018 4:07 AM
>> To: Michel Dänzer <michel at daenzer.net>; Li, Samuel
>> <Samuel.Li at amd.com>; Alex Deucher <alexdeucher at gmail.com>
>> Cc: amd-gfx list <amd-gfx at lists.freedesktop.org>
>> Subject: Re: [PATCH 1/2] drm/amdgpu: Enable scatter gather display support
>>
>> Am 08.03.2018 um 09:35 schrieb Michel Dänzer:
>>
>> On 2018-03-07 10:47 AM, Christian König wrote:
>>
>> Am 07.03.2018 um 09:42 schrieb Michel Dänzer:
>>
>> On 2018-03-06 07:23 PM, Christian König wrote:
>>
>> E.g. the last time I tested it placing things into GTT still
>> resulted in quite a performance penalty for rendering.
>>
>> FWIW, I think the penalty is most likely IOMMU related. Last time I
>> tested, I couldn't measure a big difference with IOMMU disabled.
>>
>> No, the penalty I'm talking about came from the ping/pong we did with
>> the scanout buffers.
>>
>> See when I tested this the DDX and Mesa where unmodified, so both
>> still assumed VRAM as placement for scanout BOs, but the kernel
>> forced scanout BOs into GTT for testing.
>>
>> So what happened was that on scanout we moved the VRAM BO to GTT
>>
>> and
>>
>> after unpinning it on the first command submission which used the BO
>> we moved it back to VRAM again.
>>
>> In the meantime, I've had the same idea as Marek: Can't the kernel
>> driver simply change the BO's preferred domain to GTT when scanning
>> out from it? Then it won't move back to VRAM.
>>
>> Yes, I've considered this as well.
>>
>> But I think making the decision in Mesa is the cleaner approach.
>>
>> E.g. so far we only override the placement decision of userspace for two
>> reasons:
>> 1. We where running out of memory in VRAM.
>> 2. We have a hardware restriction which makes VRAM usage mandatory.
>>
>> And even then we never adjust the placement permanently, we just
>> temporary moved the buffer where it was needed and moved it back after
>> the operation completed.
>>
>> Additional to that Mesa might want to set even more flags and/or changes
>> it's behavior. E.g. use a tilling mode which both importer and export in
>> an
>> A+A laptop understands etc...
>>
>> Regards,
>> Christian.
>>
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> amd-gfx mailing list
>>
>> amd-gfx at lists.freedesktop.org
>>
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>>
>
>