<div dir="ltr"><div dir="ltr">On Mon, 3 Jun 2019 at 13:27, Koenig, Christian <<a href="mailto:Christian.Koenig@amd.com">Christian.Koenig@amd.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_-5694667062121711517moz-cite-prefix">Am 03.06.19 um 14:21 schrieb Alex Smith:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">On Mon, 3 Jun 2019 at 11:57, Koenig, Christian <<a href="mailto:Christian.Koenig@amd.com" target="_blank">Christian.Koenig@amd.com</a>> wrote:<br>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Am 02.06.19 um 12:32 schrieb Alex Smith:<br>
> Put the uncached GTT type at a higher index than the visible VRAM type,<br>
> rather than having GTT first.<br>
><br>
> When we don't have dedicated VRAM, we don't have a non-visible VRAM<br>
> type, and the property flags for GTT and visible VRAM are identical.<br>
> According to the spec, for types with identical flags, we should give<br>
> the one with better performance a lower index.<br>
><br>
> Previously, apps which follow the spec guidance for choosing a memory<br>
> type would have picked the GTT type in preference to visible VRAM (all<br>
> Feral games will do this), and end up with lower performance.<br>
><br>
> On a Ryzen 5 2500U laptop (Raven Ridge), this improves average FPS in<br>
> the Rise of the Tomb Raider benchmark by up to ~30%. Tested a couple of<br>
> other (Feral) games and saw similar improvement on those as well.<br>
<br>
Well that patch doesn't looks like a good idea to me.<br>
<br>
Using VRAM over uncached GTT should have something between no and only <br>
minimal performance difference on APU.<br>
<br>
To make things even worse VRAM is still needed for scanout and newer <br>
laptops have only a very very low default setting (32 or 16MB). So you <br>
can end up in VRAM clashing on those systems.<br>
<br>
Can you check some kernel statistics to figure out what exactly is going <br>
on here?<br>
</blockquote>
<div><br>
</div>
What statistics should I look at?</div>
</div>
</blockquote>
<br>
First of all take a look at amdgpu_gem_info file in the debugfs directory while using GTT and match that to using VRAM. You should see a lot more of GTT ... CPU_GTT_USWC entries with the GTT variant. If the CPU_GTT_USWC flag is missing we have found the problem.<br>
<br>
If that looks ok, then take a look at the ttm_page_pool or ttm_dma_page_pool file and see how many wc/uc and wc/uc huge pages you got. Huge pages should be used for anything larger than 2MB, if not we have found the problem.<br>
<br>
If that still isn't the issue I need to take a look at the VM code again and see if we still map VRAM/GTT differently on APUs.<br></div></blockquote><div><br></div><div>OK, got around to looking at this. amdgpu_gem_info does have more USWC entries when using GTT. I've attached the output from VRAM vs GTT in case you can spot anything else in there.</div><div><br></div><div>ttm_page_pool has 9806 wc, 238 wc huge, no uc or uc huge.</div><div><br></div><div>FWIW this was from kernel 5.0.10, I just upgraded to 5.1.6 and still the same perf difference there.</div><div><br></div><div>Thanks,</div><div>Alex</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<br>
Thanks,<br>
Christian.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>Thanks,</div>
<div>Alex</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Regards,<br>
Christian.<br>
<br>
><br>
> Signed-off-by: Alex Smith <<a href="mailto:asmith@feralinteractive.com" target="_blank">asmith@feralinteractive.com</a>><br>
> ---<br>
> I noticed that the memory types advertised on my Raven laptop looked a<br>
> bit odd so played around with it and found this. I'm not sure if it is<br>
> actually expected that the performance difference between visible VRAM<br>
> and GTT is so large, seeing as it's not dedicated VRAM, but the results<br>
> are clear (and consistent, tested multiple times).<br>
> ---<br>
> src/amd/vulkan/radv_device.c | 18 +++++++++++++++---<br>
> 1 file changed, 15 insertions(+), 3 deletions(-)<br>
><br>
> diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c<br>
> index 3cf050ed220..d36ee226ebd 100644<br>
> --- a/src/amd/vulkan/radv_device.c<br>
> +++ b/src/amd/vulkan/radv_device.c<br>
> @@ -171,12 +171,11 @@ radv_physical_device_init_mem_types(struct radv_physical_device *device)<br>
> .heapIndex = vram_index,<br>
> };<br>
> }<br>
> - if (gart_index >= 0) {<br>
> + if (gart_index >= 0 && device->rad_info.has_dedicated_vram) {<br>
> device->mem_type_indices[type_count] = RADV_MEM_TYPE_GTT_WRITE_COMBINE;<br>
> device->memory_properties.memoryTypes[type_count++] = (VkMemoryType) {<br>
> .propertyFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |<br>
> - VK_MEMORY_PROPERTY_HOST_COHERENT_BIT |<br>
> - (device->rad_info.has_dedicated_vram ? 0 : VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT),<br>
> + VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,<br>
> .heapIndex = gart_index,<br>
> };<br>
> }<br>
> @@ -189,6 +188,19 @@ radv_physical_device_init_mem_types(struct radv_physical_device *device)<br>
> .heapIndex = visible_vram_index,<br>
> };<br>
> }<br>
> + if (gart_index >= 0 && !device->rad_info.has_dedicated_vram) {<br>
> + /* Put GTT after visible VRAM for GPUs without dedicated VRAM<br>
> + * as they have identical property flags, and according to the<br>
> + * spec, for types with identical flags, the one with greater<br>
> + * performance must be given a lower index. */<br>
> + device->mem_type_indices[type_count] = RADV_MEM_TYPE_GTT_WRITE_COMBINE;<br>
> + device->memory_properties.memoryTypes[type_count++] = (VkMemoryType) {<br>
> + .propertyFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT |<br>
> + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |<br>
> + VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,<br>
> + .heapIndex = gart_index,<br>
> + };<br>
> + }<br>
> if (gart_index >= 0) {<br>
> device->mem_type_indices[type_count] = RADV_MEM_TYPE_GTT_CACHED;<br>
> device->memory_properties.memoryTypes[type_count++] = (VkMemoryType) {<br>
<br>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote></div></div>