Random short freezes due to TTM buffer migrations

[Alex Deucher]

On Tue, Aug 16, 2016 at 11:58 AM, Christian König
<deathsimple at vodafone.de> wrote:
> Am 16.08.2016 um 17:50 schrieb Alex Deucher:
>>
>> On Tue, Aug 16, 2016 at 11:27 AM, Christian König
>> <deathsimple at vodafone.de> wrote:
>>>
>>> Hi Marek,
>>>
>>> I'm already working on this.
>>>
>>> My current approach is to use a custom BO manager for VRAM with TTM and
>>> so
>>> split allocations into chunks of 4MB.
>>>
>> How about handling vram in fragment size pages (64k) or does the
>> overhead get too high?
>
>
> I've added a parameter for this, but the overhead with 64k is way to much.
> We really need something bigger here.
>
> For my experiments I split VRAM allocations in chunks of 4MB, but any
> multiple of 4KB should work as well.
>
>> Or add logic to migrate to contiguous for
>> scanout or for engines without vm support.
>
>
> Well that was the easy part and is already done. Hammering out the bugs of
> the VM and migration functions turned out to be a bit more problematic.
>
>> We could reserve part of
>> the gart aperture for driver use for paged vram migration handling.
>
> Rather easily doable as well (you don't even need to reserve anything, but
> could just ask the drm MM to return a piece of address space).
>
> But I would advise against it, cause we don't really know how UVD/VCE/DCE
> would react to that.

I just meant to use part of it to make migrations easier, not to use
it for engines without VM.  E.g., in order to provide a contiguous
view of the pages that you want to migrate to/from vram.  But thinking
about it more, the disadvantages out-weigh the advantages.

Alex

>
> Christian.
>
>
>>
>> Alex
>>
>>> Large BOs are still swapped out as one, but it makes it much more likely
>>> to
>>> that you can allocate 1/2 of VRAM as one buffer.
>>>
>>> Give me till the end of the week to finish this and then we can test if
>>> that's sufficient or if we need to do more.
>>>
>>> Regards,
>>> Christian.
>>>
>>>
>>> Am 16.08.2016 um 16:33 schrieb Marek Olšák:
>>>>
>>>> Hi,
>>>>
>>>> I'm seeing random temporary freezes (up to 2 seconds) under memory
>>>> pressure. Before I describe the exact circumstances, I'd like to say
>>>> that this is a serious issue affecting playability of certain AAA
>>>> Linux games.
>>>>
>>>> In order to reproduce this, an application should:
>>>> - allocate a few very large buffers (256-512 MB per buffer)
>>>> - allocate more memory than there is available VRAM. The issue also
>>>> occurs (but at a lower frequency) if the app needs only 80% of VRAM.
>>>>
>>>> Example: ttm_bo_validate needs to migrate a 512 MB buffer. The total
>>>> size of moved memory for that call can be as high as 1.5 GB. This is
>>>> always followed by a big temporary drop in VRAM usage.
>>>>
>>>> The game I'm testing needs 3.4 GB of VRAM.
>>>>
>>>> Setups:
>>>> Tonga - 2 GB: It's nearly unplayable, because freezes occur too often.
>>>> Fiji - 4 GB: There is one freeze at the beginning (which is annoying
>>>> too), after that it's smooth.
>>>>
>>>> So even 4 GB is not enough.
>>>>
>>>> Workarounds:
>>>> - Split buffers into smaller pieces in the kernel. It's not necessary
>>>> to manage memory at page granularity (64KB). Splitting buffers into
>>>> 16MB-large pieces might not be optimal but it would be a significant
>>>> improvement.
>>>> - Or do the same in Mesa. This would prevent inter-process and
>>>> inter-API buffer sharing for split buffers (DRI, OpenCL), but we would
>>>> at least verify how much the situation improves.
>>>>
>>>> Other issues sharing the same cause:
>>>> - Allocations requesting 1/3 or more VRAM have a high chance of
>>>> failing. It's generally not possible to allocate 1/2 or more VRAM as
>>>> one buffer.
>>>>
>>>> Comments welcome,
>>>>
>>>> Marek
>>>> _______________________________________________
>>>> amd-gfx mailing list
>>>> amd-gfx at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>>
>>>
>>>
>>> _______________________________________________
>>> amd-gfx mailing list
>>> amd-gfx at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
>
>