<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0"> > From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> Informal commit message for now.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> I got a bit impatient and curious to see if the idea we discussed would</div> <div class="ContentPasted0">> work so sketched something out. I think it is what I was describing back</div> <div class="ContentPasted0">> then..</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> So high level idea is to teach the driver what caching modes are hidden</div> <div class="ContentPasted0">> behind PAT indices. Given you already had that in static tables, if we</div> <div class="ContentPasted0">> just turn the tables a bit around and add a driver abstraction of caching</div> <div class="ContentPasted0">> modes this is what happens:</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> * We can lose the ugly runtime i915_gem_get_pat_index.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> * We can have a smarter i915_gem_object_has_cache_level, which now can</div> <div class="ContentPasted0">> use the above mentioned table to understand the caching modes and so</div> <div class="ContentPasted0">> does not have to pessimistically return true for _any_ input when user</div> <div class="ContentPasted0">> has set the PAT index. This may improve things even for MTL.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> * We can simplify the debugfs printout to be platform agnostic.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> * We are perhaps opening the door to un-regress the dodgy addition</div> <div class="ContentPasted0">> made to i915_gem_object_can_bypass_llc? See QQQ/FIXME in the patch.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> I hope I did not forget anything, but anyway, please have a read and see</div> <div class="ContentPasted0">> what you think. I think it has potential.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> Proper commit message can come later.</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com></div> <div class="ContentPasted0">> Cc: Fei Yang <fei.yang@intel.com></div> <div class="ContentPasted0">> ---</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/Makefile | 1 +</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/display/intel_dpt.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/display/intel_fb_pin.c | 2 +-</div> <div class="ContentPasted0">> .../drm/i915/display/intel_plane_initial.c | 4 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_domain.c | 66 +++++-----</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_domain.h | 7 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 13 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_internal.c | 6 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_mman.c | 10 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_object.c | 116 +++++++++--------</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_object.h | 9 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/gem/i915_gem_object_types.h | 117 +++---------------</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 10 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 13 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 43 ++++---</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 2 +-</div> <div class="ContentPasted0">> .../drm/i915/gem/selftests/huge_gem_object.c | 6 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/gem/selftests/huge_pages.c | 8 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 4 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 19 +--</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_ggtt.c | 33 ++---</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c | 4 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_gtt.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_gtt.h | 3 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_migrate.c | 11 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_ppgtt.c | 6 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/gt/intel_ring_submission.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/intel_timeline.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/selftest_migrate.c | 9 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/selftest_reset.c | 14 +--</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/selftest_timeline.c | 1 +</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/selftest_tlb.c | 5 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/gt/selftest_workarounds.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/uc/intel_guc.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 8 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_cache.c | 72 +++++++++++</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_cache.h | 49 ++++++++</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_debugfs.c | 65 +++-------</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_driver.c | 5 +</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_drv.h | 3 +</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_gem.c | 21 +---</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_gpu_error.c | 7 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_pci.c | 83 +++++++------</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/i915_perf.c | 2 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/intel_device_info.h | 6 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/selftests/i915_gem.c | 5 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/selftests/i915_gem_evict.c | 8 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 13 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/selftests/igt_spinner.c | 2 +-</div> <div class="ContentPasted0">> .../drm/i915/selftests/intel_memory_region.c | 4 +-</div> <div class="ContentPasted0">> .../gpu/drm/i915/selftests/mock_gem_device.c | 12 +-</div> <div class="ContentPasted0">> drivers/gpu/drm/i915/selftests/mock_region.c | 2 +-</div> <div class="ContentPasted0">> 54 files changed, 440 insertions(+), 485 deletions(-)</div> <div class="ContentPasted0">> create mode 100644 drivers/gpu/drm/i915/i915_cache.c</div> <div class="ContentPasted0">> create mode 100644 drivers/gpu/drm/i915/i915_cache.h</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile</div> <div class="ContentPasted0">> index 2be9dd960540..2c3da8f0c78e 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/Makefile</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/Makefile</div> <div class="ContentPasted0">> @@ -30,6 +30,7 @@ subdir-ccflags-y += -I$(srctree)/$(src)</div> <div class="ContentPasted0">> # core driver code</div> <div class="ContentPasted0">> i915-y += i915_driver.o \</div> <div class="ContentPasted0">> i915_drm_client.o \</div> <div class="ContentPasted0">> + i915_cache.o \</div> <div class="ContentPasted0">> i915_config.o \</div> <div class="ContentPasted0">> i915_getparam.o \</div> <div class="ContentPasted0">> i915_ioctl.o \</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c</div> <div class="ContentPasted0">> index 7c5fddb203ba..5baf0d27a288 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/display/intel_dpt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/display/intel_dpt.c</div> <div class="ContentPasted0">> @@ -268,7 +268,7 @@ intel_dpt_create(struct intel_framebuffer *fb)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> ret = i915_gem_object_lock_interruptible(dpt_obj, NULL);</div> <div class="ContentPasted0">> if (!ret) {</div> <div class="ContentPasted0">> - ret = i915_gem_object_set_cache_level(dpt_obj, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + ret = i915_gem_object_set_cache(dpt_obj, I915_CACHE(UC));</div> <div class="ContentPasted0">> i915_gem_object_unlock(dpt_obj);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> if (ret) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/display/intel_fb_pin.c b/drivers/gpu/drm/i915/display/intel_fb_pin.c</div> <div class="ContentPasted0">> index fffd568070d4..dcf54b354a74 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/display/intel_fb_pin.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/display/intel_fb_pin.c</div> <div class="ContentPasted0">> @@ -66,7 +66,7 @@ intel_pin_fb_obj_dpt(struct drm_framebuffer *fb,</div> <div class="ContentPasted0">> continue;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ret = i915_gem_object_set_cache_level(obj, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + ret = i915_gem_object_set_cache(obj, I915_CACHE(UC));</div> <div class="ContentPasted0">> if (ret)</div> <div class="ContentPasted0">> continue;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/display/intel_plane_initial.c b/drivers/gpu/drm/i915/display/intel_plane_initial.c</div> <div class="ContentPasted0">> index 736072a8b2b0..9988f6562392 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/display/intel_plane_initial.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/display/intel_plane_initial.c</div> <div class="ContentPasted0">> @@ -121,8 +121,8 @@ initial_plane_vma(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> * cache_level during fbdev initialization. The</div> <div class="ContentPasted0">> * unbind there would get stuck waiting for rcu.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, HAS_WT(i915) ?</div> <div class="ContentPasted0">> - I915_CACHE_WT : I915_CACHE_NONE);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, HAS_WT(i915) ? I915_CACHE(WT) :</div> <div class="ContentPasted0">> + I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> switch (plane_config->tiling) {</div> <div class="ContentPasted0">> case I915_TILING_NONE:</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c</div> <div class="ContentPasted0">> index dfaaa8b66ac3..f899da2c622a 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c</div> <div class="ContentPasted0">> @@ -8,6 +8,7 @@</div> <div class="ContentPasted0">> #include "display/intel_frontbuffer.h"</div> <div class="ContentPasted0">> #include "gt/intel_gt.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> #include "i915_drv.h"</div> <div class="ContentPasted0">> #include "i915_gem_clflush.h"</div> <div class="ContentPasted0">> #include "i915_gem_domain.h"</div> <div class="ContentPasted0">> @@ -27,15 +28,8 @@ static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> if (IS_DGFX(i915))</div> <div class="ContentPasted0">> return false;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * For objects created by userspace through GEM_CREATE with pat_index</div> <div class="ContentPasted0">> - * set by set_pat extension, i915_gem_object_has_cache_level() will</div> <div class="ContentPasted0">> - * always return true, because the coherency of such object is managed</div> <div class="ContentPasted0">> - * by userspace. Othereise the call here would fall back to checking</div> <div class="ContentPasted0">> - * whether the object is un-cached or write-through.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - return !(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) ||</div> <div class="ContentPasted0">> - i915_gem_object_has_cache_level(obj, I915_CACHE_WT));</div> <div class="ContentPasted0">> + return i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_UC) != 1 &&</div> <div class="ContentPasted0">> + i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_WT) != 1;</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">This logic was changed for objects with pat index set by user here. It used to return false</div> <div class="ContentPasted0">regardless the cache mode. But now it returns true if its cache mode is neither UC nor WT. </div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> @@ -255,9 +249,9 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /**</div> <div class="ContentPasted0">> - * i915_gem_object_set_cache_level - Changes the cache-level of an object across all VMA.</div> <div class="ContentPasted0">> + * i915_gem_object_set_cache - Changes the cache-level of an object across all VMA.</div> <div class="ContentPasted0">> * @obj: object to act on</div> <div class="ContentPasted0">> - * @cache_level: new cache level to set for the object</div> <div class="ContentPasted0">> + * @cache: new caching mode to set for the object</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * After this function returns, the object will be in the new cache-level</div> <div class="ContentPasted0">> * across all GTT and the contents of the backing storage will be coherent,</div> <div class="ContentPasted0">> @@ -269,19 +263,28 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)</div> <div class="ContentPasted0">> * cache coherency) and all non-MOCS GPU access will also be uncached so</div> <div class="ContentPasted0">> * that all direct access to the scanout remains coherent.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> -int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - enum i915_cache_level cache_level)</div> <div class="ContentPasted0">> +int i915_gem_object_set_cache(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> + i915_cache_t cache)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = to_i915(obj->base.dev);</div> <div class="ContentPasted0">> int ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * For objects created by userspace through GEM_CREATE with pat_index</div> <div class="ContentPasted0">> - * set by set_pat extension, simply return 0 here without touching</div> <div class="ContentPasted0">> - * the cache setting, because such objects should have an immutable</div> <div class="ContentPasted0">> - * cache setting by desgin and always managed by userspace.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - if (i915_gem_object_has_cache_level(obj, cache_level))</div> <div class="ContentPasted0">> + ret = i915_cache_find_pat(i915, cache);</div> <div class="ContentPasted0">> + if (ret < 0) {</div> <div class="ContentPasted0">> + struct drm_printer p =</div> <div class="ContentPasted0">> + drm_err_printer("Attempting to use unknown caching mode ");</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + i915_cache_print(&p, cache);</div> <div class="ContentPasted0">> + drm_puts(&p, "!\n");</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + return -EINVAL;</div> <div class="ContentPasted0">> + } else if (ret == obj->pat_index) {</div> <div class="ContentPasted0">> return 0;</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">We will have to do this conversion and checking again later in this</div> <div class="ContentPasted0">function when calling i915_gem_object_set_cache_coherency().</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">My thought was to simply remove the use of cache_level/cache and replace</div> <div class="ContentPasted0">it with pat_idex. Conversion from cache modes to pat index should be done</div> <div class="ContentPasted0">before calling any function used to consume cache_level/cache.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> + } else if (obj->pat_set_by_user) {</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">Shouldn't this if-statement be placed at the beginning of this function? the original</div> <div class="ContentPasted0">i915_gem_object_has_cache_level() would return true if (obj->pat_set_by_user), so the</div> <div class="ContentPasted0">function returns right away.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> + drm_notice_once(&i915->drm,</div> <div class="ContentPasted0">> + "Attempting to change caching mode on an object with fixed PAT!\n");</div> <div class="ContentPasted0">> + return -EINVAL;</div> <div class="ContentPasted0">> + }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> ret = i915_gem_object_wait(obj,</div> <div class="ContentPasted0">> I915_WAIT_INTERRUPTIBLE |</div> <div class="ContentPasted0">> @@ -291,7 +294,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> return ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* Always invalidate stale cachelines */</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, cache);</div> <div class="ContentPasted0">> obj->cache_dirty = true;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* The cache-level will be applied when each vma is rebound. */</div> <div class="ContentPasted0">> @@ -326,10 +329,9 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data,</div> <div class="ContentPasted0">> goto out;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (i915_gem_object_has_cache_level(obj, I915_CACHE_LLC) ||</div> <div class="ContentPasted0">> - i915_gem_object_has_cache_level(obj, I915_CACHE_L3_LLC))</div> <div class="ContentPasted0">> + if (i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_WB))</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">This looks wrong, at least for MTL. Prior to MTL the GPU automatically snoop</div> <div class="ContentPasted0">CPU cache, but from MTL onward you have to specify if WB or WB + 1-WAY COH is</div> <div class="ContentPasted0">needed. And for KMD, cacheable mode means WB + 1-WAY COH for MTL to keep the</div> <div class="ContentPasted0">behavior consistent.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">This used to be taken care of by i915_gem_get_pat_index() call. With that being</div> <div class="ContentPasted0">replaced by i915_cache_find_pat(), you would need to do something there. But,</div> <div class="ContentPasted0">without cachelevel_to_pat[], you might end up hard coding something directly in</div> <div class="ContentPasted0">the function, and that is platform dependent. hmm..., I don't really like this idea.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">That's why I commented in v1 that we should use INTEL_INFO(i915)->pat_uc/wb/wt</div> <div class="ContentPasted0">instead of enum i915_cache_level or i915_cache_t.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> args->caching = I915_CACHING_CACHED;</div> <div class="ContentPasted0">> - else if (i915_gem_object_has_cache_level(obj, I915_CACHE_WT))</div> <div class="ContentPasted0">> + else if (i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_WT))</div> <div class="ContentPasted0">> args->caching = I915_CACHING_DISPLAY;</div> <div class="ContentPasted0">> else</div> <div class="ContentPasted0">> args->caching = I915_CACHING_NONE;</div> <div class="ContentPasted0">> @@ -344,7 +346,7 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,</div> <div class="ContentPasted0">> struct drm_i915_private *i915 = to_i915(dev);</div> <div class="ContentPasted0">> struct drm_i915_gem_caching *args = data;</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> - enum i915_cache_level level;</div> <div class="ContentPasted0">> + i915_cache_t cache;</div> <div class="ContentPasted0">> int ret = 0;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (IS_DGFX(i915))</div> <div class="ContentPasted0">> @@ -355,7 +357,7 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> switch (args->caching) {</div> <div class="ContentPasted0">> case I915_CACHING_NONE:</div> <div class="ContentPasted0">> - level = I915_CACHE_NONE;</div> <div class="ContentPasted0">> + cache = I915_CACHE(UC);</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">For code like this, my thought was</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">- level = I915_CACHE_NONE;</div> <div class="ContentPasted0">+ pat_index = INTEL_INFO(i915)->pat_uc;</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">And later the set_cache call should take pat_index as argument instead of cache mode.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> case I915_CACHING_CACHED:</div> <div class="ContentPasted0">> /*</div> <div class="ContentPasted0">> @@ -367,10 +369,10 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,</div> <div class="ContentPasted0">> if (!HAS_LLC(i915) && !HAS_SNOOP(i915))</div> <div class="ContentPasted0">> return -ENODEV;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - level = I915_CACHE_LLC;</div> <div class="ContentPasted0">> + cache = I915_CACHE(WB);</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">- level = I915_CACHE_LLC;</div> <div class="ContentPasted0">+ pat_index = INTEL_INFO(i915)->pat_wb;</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">This should take care of the WB + 1-WAY COH issue for MTL mentioned above,</div> <div class="ContentPasted0">assuming the i915_cache_init() set pat_wb properly, and the</div> <div class="ContentPasted0">i915_gem_object_set_cache() consumes pat_index instead of cache mode.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> case I915_CACHING_DISPLAY:</div> <div class="ContentPasted0">> - level = HAS_WT(i915) ? I915_CACHE_WT : I915_CACHE_NONE;</div> <div class="ContentPasted0">> + cache = HAS_WT(i915) ? I915_CACHE(WT) : I915_CACHE(UC);</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> default:</div> <div class="ContentPasted0">> return -EINVAL;</div> <div class="ContentPasted0">> @@ -409,7 +411,7 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,</div> <div class="ContentPasted0">> if (ret)</div> <div class="ContentPasted0">> goto out;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ret = i915_gem_object_set_cache_level(obj, level);</div> <div class="ContentPasted0">> + ret = i915_gem_object_set_cache(obj, cache);</div> <div class="ContentPasted0">> i915_gem_object_unlock(obj);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> out:</div> <div class="ContentPasted0">> @@ -448,9 +450,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> * of uncaching, which would allow us to flush all the LLC-cached data</div> <div class="ContentPasted0">> * with that bit in the PTE to main memory with just one PIPE_CONTROL.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - ret = i915_gem_object_set_cache_level(obj,</div> <div class="ContentPasted0">> - HAS_WT(i915) ?</div> <div class="ContentPasted0">> - I915_CACHE_WT : I915_CACHE_NONE);</div> <div class="ContentPasted0">> + ret = i915_gem_object_set_cache(obj,</div> <div class="ContentPasted0">> + HAS_WT(i915) ?</div> <div class="ContentPasted0">> + I915_CACHE(WT) : I915_CACHE(UC));</div> <div class="ContentPasted0">> if (ret)</div> <div class="ContentPasted0">> return ERR_PTR(ret);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.h b/drivers/gpu/drm/i915/gem/i915_gem_domain.h</div> <div class="ContentPasted0">> index 9622df962bfc..d07926758b36 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.h</div> <div class="ContentPasted0">> @@ -6,10 +6,11 @@</div> <div class="ContentPasted0">> #ifndef __I915_GEM_DOMAIN_H__</div> <div class="ContentPasted0">> #define __I915_GEM_DOMAIN_H__</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> struct drm_i915_gem_object;</div> <div class="ContentPasted0">> -enum i915_cache_level;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - enum i915_cache_level cache_level);</div> <div class="ContentPasted0">> +int i915_gem_object_set_cache(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> + i915_cache_t cache);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #endif /* __I915_GEM_DOMAIN_H__ */</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c</div> <div class="ContentPasted0">> index d3208a325614..ee85221fa6eb 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c</div> <div class="ContentPasted0">> @@ -640,15 +640,9 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache,</div> <div class="ContentPasted0">> if (DBG_FORCE_RELOC == FORCE_GTT_RELOC)</div> <div class="ContentPasted0">> return false;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * For objects created by userspace through GEM_CREATE with pat_index</div> <div class="ContentPasted0">> - * set by set_pat extension, i915_gem_object_has_cache_level() always</div> <div class="ContentPasted0">> - * return true, otherwise the call would fall back to checking whether</div> <div class="ContentPasted0">> - * the object is un-cached.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> return (cache->has_llc ||</div> <div class="ContentPasted0">> obj->cache_dirty ||</div> <div class="ContentPasted0">> - !i915_gem_object_has_cache_level(obj, I915_CACHE_NONE));</div> <div class="ContentPasted0">> + i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_UC) != 1);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static int eb_reserve_vma(struct i915_execbuffer *eb,</div> <div class="ContentPasted0">> @@ -1329,10 +1323,7 @@ static void *reloc_iomap(struct i915_vma *batch,</div> <div class="ContentPasted0">> if (drm_mm_node_allocated(&cache->node)) {</div> <div class="ContentPasted0">> ggtt->vm.insert_page(&ggtt->vm,</div> <div class="ContentPasted0">> i915_gem_object_get_dma_address(obj, page),</div> <div class="ContentPasted0">> - offset,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(ggtt->vm.i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + offset, eb->i915->pat_uc, 0);</div> <div class="ContentPasted0">> } else {</div> <div class="ContentPasted0">> offset += page << PAGE_SHIFT;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_internal.c b/drivers/gpu/drm/i915/gem/i915_gem_internal.c</div> <div class="ContentPasted0">> index 6bc26b4b06b8..7e7bfe7da76f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_internal.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_internal.c</div> <div class="ContentPasted0">> @@ -142,7 +142,6 @@ __i915_gem_object_create_internal(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> static struct lock_class_key lock_class;</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> GEM_BUG_ON(!size);</div> <div class="ContentPasted0">> GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE));</div> <div class="ContentPasted0">> @@ -170,8 +169,9 @@ __i915_gem_object_create_internal(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, HAS_LLC(i915) ?</div> <div class="ContentPasted0">> + I915_CACHE(WB) :</div> <div class="ContentPasted0">> + I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return obj;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c</div> <div class="ContentPasted0">> index aa4d842d4c5a..5e21aedb02d2 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c</div> <div class="ContentPasted0">> @@ -386,13 +386,11 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf)</div> <div class="ContentPasted0">> /*</div> <div class="ContentPasted0">> * For objects created by userspace through GEM_CREATE with pat_index</div> <div class="ContentPasted0">> * set by set_pat extension, coherency is managed by userspace, make</div> <div class="ContentPasted0">> - * sure we don't fail handling the vm fault by calling</div> <div class="ContentPasted0">> - * i915_gem_object_has_cache_level() which always return true for such</div> <div class="ContentPasted0">> - * objects. Otherwise this helper function would fall back to checking</div> <div class="ContentPasted0">> - * whether the object is un-cached.</div> <div class="ContentPasted0">> + * sure we don't fail handling the vm fault by making sure that we</div> <div class="ContentPasted0">> + * know the object is uncached or that we have LLC.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - if (!(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) ||</div> <div class="ContentPasted0">> - HAS_LLC(i915))) {</div> <div class="ContentPasted0">> + if (i915_gem_object_has_cache_mode(obj, I915_CACHE_MODE_UC) != 1 &&</div> <div class="ContentPasted0">> + !HAS_LLC(i915)) {</div> <div class="ContentPasted0">> ret = -EFAULT;</div> <div class="ContentPasted0">> goto err_unpin;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c</div> <div class="ContentPasted0">> index d77a375871b7..2d3a58c62122 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c</div> <div class="ContentPasted0">> @@ -45,33 +45,6 @@ static struct kmem_cache *slab_objects;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct drm_gem_object_funcs i915_gem_object_funcs;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> - enum i915_cache_level level)</div> <div class="ContentPasted0">> -{</div> <div class="ContentPasted0">> - if (drm_WARN_ON(&i915->drm, level >= I915_MAX_CACHE_LEVEL))</div> <div class="ContentPasted0">> - return 0;</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> - return INTEL_INFO(i915)->cachelevel_to_pat[level];</div> <div class="ContentPasted0">> -}</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> -bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - enum i915_cache_level lvl)</div> <div class="ContentPasted0">> -{</div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * In case the pat_index is set by user space, this kernel mode</div> <div class="ContentPasted0">> - * driver should leave the coherency to be managed by user space,</div> <div class="ContentPasted0">> - * simply return true here.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - if (obj->pat_set_by_user)</div> <div class="ContentPasted0">> - return true;</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * Otherwise the pat_index should have been converted from cache_level</div> <div class="ContentPasted0">> - * so that the following comparison is valid.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - return obj->pat_index == i915_gem_get_pat_index(obj_to_i915(obj), lvl);</div> <div class="ContentPasted0">> -}</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> struct drm_i915_gem_object *i915_gem_object_alloc(void)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> @@ -144,30 +117,70 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> dma_resv_fini(&obj->base._resv);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +int i915_gem_object_has_cache_mode(const struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> + enum i915_cache_mode mode)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = obj_to_i915(obj);</div> <div class="ContentPasted0">> + i915_cache_t cache = INTEL_INFO(i915)->cache_modes[obj->pat_index];</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + if (I915_CACHE_MODE(cache) == mode)</div> <div class="ContentPasted0">> + return 1;</div> <div class="ContentPasted0">> + else if (obj->pat_set_by_user)</div> <div class="ContentPasted0">> + return -1; /* Unknown, callers should assume no. */</div> <div class="ContentPasted0">> + else</div> <div class="ContentPasted0">> + return 0;</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +static void __i915_gem_object_update_coherency(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = obj_to_i915(obj);</div> <div class="ContentPasted0">> + i915_cache_t cache = INTEL_INFO(i915)->cache_modes[obj->pat_index];</div> <div class="ContentPasted0">> + unsigned int mode = I915_CACHE_MODE(cache);</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + if (mode == I915_CACHE_MODE_UNKNOWN) {</div> <div class="ContentPasted0">> + drm_notice_once(&i915->drm, "Undefined PAT index %u!\n",</div> <div class="ContentPasted0">> + obj->pat_index);</div> <div class="ContentPasted0">> + obj->cache_coherent = 0;</div> <div class="ContentPasted0">> + } else if (mode != I915_CACHE_MODE_UC) {</div> <div class="ContentPasted0">> + obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ |</div> <div class="ContentPasted0">> + I915_BO_CACHE_COHERENT_FOR_WRITE;</div> <div class="ContentPasted0">> + } else if (HAS_LLC(i915)) {</div> <div class="ContentPasted0">> + obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ;</div> <div class="ContentPasted0">> + } else {</div> <div class="ContentPasted0">> + obj->cache_coherent = 0;</div> <div class="ContentPasted0">> + }</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + obj->cache_dirty =</div> <div class="ContentPasted0">> + !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) &&</div> <div class="ContentPasted0">> + !IS_DGFX(i915);</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> /**</div> <div class="ContentPasted0">> * i915_gem_object_set_cache_coherency - Mark up the object's coherency levels</div> <div class="ContentPasted0">> - * for a given cache_level</div> <div class="ContentPasted0">> + * for a given caching mode</div> <div class="ContentPasted0">> * @obj: #drm_i915_gem_object</div> <div class="ContentPasted0">> - * @cache_level: cache level</div> <div class="ContentPasted0">> + * @cache: cache mode</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - unsigned int cache_level)</div> <div class="ContentPasted0">> + i915_cache_t cache)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> - struct drm_i915_private *i915 = to_i915(obj->base.dev);</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = obj_to_i915(obj);</div> <div class="ContentPasted0">> + int found;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - obj->pat_index = i915_gem_get_pat_index(i915, cache_level);</div> <div class="ContentPasted0">> + found = i915_cache_find_pat(i915, cache);</div> <div class="ContentPasted0">> + if (found < 0) {</div> <div class="ContentPasted0">> + struct drm_printer p =</div> <div class="ContentPasted0">> + drm_err_printer("Unknown cache mode ");</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (cache_level != I915_CACHE_NONE)</div> <div class="ContentPasted0">> - obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ |</div> <div class="ContentPasted0">> - I915_BO_CACHE_COHERENT_FOR_WRITE);</div> <div class="ContentPasted0">> - else if (HAS_LLC(i915))</div> <div class="ContentPasted0">> - obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ;</div> <div class="ContentPasted0">> - else</div> <div class="ContentPasted0">> - obj->cache_coherent = 0;</div> <div class="ContentPasted0">> + i915_cache_print(&p, cache);</div> <div class="ContentPasted0">> + drm_puts(&p, "!\n");</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - obj->cache_dirty =</div> <div class="ContentPasted0">> - !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) &&</div> <div class="ContentPasted0">> - !IS_DGFX(i915);</div> <div class="ContentPasted0">> + found = i915->pat_uc;</div> <div class="ContentPasted0">> + }</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + obj->pat_index = found;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + __i915_gem_object_update_coherency(obj);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /**</div> <div class="ContentPasted0">> @@ -181,24 +194,18 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> unsigned int pat_index)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> - struct drm_i915_private *i915 = to_i915(obj->base.dev);</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = obj_to_i915(obj);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (obj->pat_index == pat_index)</div> <div class="ContentPasted0">> return;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> + if (drm_WARN_ON_ONCE(&i915->drm,</div> <div class="ContentPasted0">> + pat_index > INTEL_INFO(i915)->max_pat_index))</div> <div class="ContentPasted0">> + return;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> obj->pat_index = pat_index;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (pat_index != i915_gem_get_pat_index(i915, I915_CACHE_NONE))</div> <div class="ContentPasted0">> - obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ |</div> <div class="ContentPasted0">> - I915_BO_CACHE_COHERENT_FOR_WRITE);</div> <div class="ContentPasted0">> - else if (HAS_LLC(i915))</div> <div class="ContentPasted0">> - obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ;</div> <div class="ContentPasted0">> - else</div> <div class="ContentPasted0">> - obj->cache_coherent = 0;</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> - obj->cache_dirty =</div> <div class="ContentPasted0">> - !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) &&</div> <div class="ContentPasted0">> - !IS_DGFX(i915);</div> <div class="ContentPasted0">> + __i915_gem_object_update_coherency(obj);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> @@ -215,6 +222,7 @@ bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> /*</div> <div class="ContentPasted0">> * Always flush cache for UMD objects at creation time.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> + /* QQQ/FIXME why? avoidable performance penalty? */</div> <div class="ContentPasted0">> if (obj->pat_set_by_user)</div> <div class="ContentPasted0">> return true;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h</div> <div class="ContentPasted0">> index 884a17275b3a..c852e59aeef9 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h</div> <div class="ContentPasted0">> @@ -13,6 +13,7 @@</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #include "display/intel_frontbuffer.h"</div> <div class="ContentPasted0">> #include "intel_memory_region.h"</div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> #include "i915_gem_object_types.h"</div> <div class="ContentPasted0">> #include "i915_gem_gtt.h"</div> <div class="ContentPasted0">> #include "i915_gem_ww.h"</div> <div class="ContentPasted0">> @@ -32,10 +33,6 @@ static inline bool i915_gem_object_size_2big(u64 size)</div> <div class="ContentPasted0">> return false;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> - enum i915_cache_level level);</div> <div class="ContentPasted0">> -bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - enum i915_cache_level lvl);</div> <div class="ContentPasted0">> void i915_gem_init__objects(struct drm_i915_private *i915);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> void i915_objects_module_exit(void);</div> <div class="ContentPasted0">> @@ -764,8 +761,10 @@ int i915_gem_object_wait_moving_fence(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> bool intr);</div> <div class="ContentPasted0">> bool i915_gem_object_has_unknown_state(struct drm_i915_gem_object *obj);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +int i915_gem_object_has_cache_mode(const struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> + enum i915_cache_mode mode);</div> <div class="ContentPasted0">> void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> - unsigned int cache_level);</div> <div class="ContentPasted0">> + i915_cache_t cache);</div> <div class="ContentPasted0">> void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> unsigned int pat_index);</div> <div class="ContentPasted0">> bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h</div> <div class="ContentPasted0">> index 8de2b91b3edf..a136d137b980 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h</div> <div class="ContentPasted0">> @@ -14,6 +14,7 @@</div> <div class="ContentPasted0">> #include <uapi/drm/i915_drm.h></div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #include "i915_active.h"</div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> #include "i915_selftest.h"</div> <div class="ContentPasted0">> #include "i915_vma_resource.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -116,93 +117,6 @@ struct drm_i915_gem_object_ops {</div> <div class="ContentPasted0">> const char *name; /* friendly name for debug, e.g. lockdep classes */</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -/**</div> <div class="ContentPasted0">> - * enum i915_cache_level - The supported GTT caching values for system memory</div> <div class="ContentPasted0">> - * pages.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * These translate to some special GTT PTE bits when binding pages into some</div> <div class="ContentPasted0">> - * address space. It also determines whether an object, or rather its pages are</div> <div class="ContentPasted0">> - * coherent with the GPU, when also reading or writing through the CPU cache</div> <div class="ContentPasted0">> - * with those pages.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Userspace can also control this through struct drm_i915_gem_caching.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> -enum i915_cache_level {</div> <div class="ContentPasted0">> - /**</div> <div class="ContentPasted0">> - * @I915_CACHE_NONE:</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * GPU access is not coherent with the CPU cache. If the cache is dirty</div> <div class="ContentPasted0">> - * and we need the underlying pages to be coherent with some later GPU</div> <div class="ContentPasted0">> - * access then we need to manually flush the pages.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * On shared LLC platforms reads and writes through the CPU cache are</div> <div class="ContentPasted0">> - * still coherent even with this setting. See also</div> <div class="ContentPasted0">> - * &drm_i915_gem_object.cache_coherent for more details. Due to this we</div> <div class="ContentPasted0">> - * should only ever use uncached for scanout surfaces, otherwise we end</div> <div class="ContentPasted0">> - * up over-flushing in some places.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * This is the default on non-LLC platforms.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - I915_CACHE_NONE = 0,</div> <div class="ContentPasted0">> - /**</div> <div class="ContentPasted0">> - * @I915_CACHE_LLC:</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * GPU access is coherent with the CPU cache. If the cache is dirty,</div> <div class="ContentPasted0">> - * then the GPU will ensure that access remains coherent, when both</div> <div class="ContentPasted0">> - * reading and writing through the CPU cache. GPU writes can dirty the</div> <div class="ContentPasted0">> - * CPU cache.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Not used for scanout surfaces.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Applies to both platforms with shared LLC(HAS_LLC), and snooping</div> <div class="ContentPasted0">> - * based platforms(HAS_SNOOP).</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * This is the default on shared LLC platforms. The only exception is</div> <div class="ContentPasted0">> - * scanout objects, where the display engine is not coherent with the</div> <div class="ContentPasted0">> - * CPU cache. For such objects I915_CACHE_NONE or I915_CACHE_WT is</div> <div class="ContentPasted0">> - * automatically applied by the kernel in pin_for_display, if userspace</div> <div class="ContentPasted0">> - * has not done so already.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - I915_CACHE_LLC,</div> <div class="ContentPasted0">> - /**</div> <div class="ContentPasted0">> - * @I915_CACHE_L3_LLC:</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Explicitly enable the Gfx L3 cache, with coherent LLC.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * The Gfx L3 sits between the domain specific caches, e.g</div> <div class="ContentPasted0">> - * sampler/render caches, and the larger LLC. LLC is coherent with the</div> <div class="ContentPasted0">> - * GPU, but L3 is only visible to the GPU, so likely needs to be flushed</div> <div class="ContentPasted0">> - * when the workload completes.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Not used for scanout surfaces.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Only exposed on some gen7 + GGTT. More recent hardware has dropped</div> <div class="ContentPasted0">> - * this explicit setting, where it should now be enabled by default.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - I915_CACHE_L3_LLC,</div> <div class="ContentPasted0">> - /**</div> <div class="ContentPasted0">> - * @I915_CACHE_WT:</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Write-through. Used for scanout surfaces.</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * The GPU can utilise the caches, while still having the display engine</div> <div class="ContentPasted0">> - * be coherent with GPU writes, as a result we don't need to flush the</div> <div class="ContentPasted0">> - * CPU caches when moving out of the render domain. This is the default</div> <div class="ContentPasted0">> - * setting chosen by the kernel, if supported by the HW, otherwise we</div> <div class="ContentPasted0">> - * fallback to I915_CACHE_NONE. On the CPU side writes through the CPU</div> <div class="ContentPasted0">> - * cache still need to be flushed, to remain coherent with the display</div> <div class="ContentPasted0">> - * engine.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - I915_CACHE_WT,</div> <div class="ContentPasted0">> - /**</div> <div class="ContentPasted0">> - * @I915_MAX_CACHE_LEVEL:</div> <div class="ContentPasted0">> - *</div> <div class="ContentPasted0">> - * Mark the last entry in the enum. Used for defining cachelevel_to_pat</div> <div class="ContentPasted0">> - * array for cache_level to pat translation table.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - I915_MAX_CACHE_LEVEL,</div> <div class="ContentPasted0">> -};</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> enum i915_map_type {</div> <div class="ContentPasted0">> I915_MAP_WB = 0,</div> <div class="ContentPasted0">> I915_MAP_WC,</div> <div class="ContentPasted0">> @@ -409,9 +323,7 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * Check for @pat_set_by_user to find out if an object has pat index set</div> <div class="ContentPasted0">> * by userspace. The ioctl's to change cache settings have also been</div> <div class="ContentPasted0">> * disabled for the objects with pat index set by userspace. Please don't</div> <div class="ContentPasted0">> - * assume @cache_coherent having the flags set as describe here. A helper</div> <div class="ContentPasted0">> - * function i915_gem_object_has_cache_level() provides one way to bypass</div> <div class="ContentPasted0">> - * the use of this field.</div> <div class="ContentPasted0">> + * assume @cache_coherent having the flags set as describe here.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * Track whether the pages are coherent with the GPU if reading or</div> <div class="ContentPasted0">> * writing through the CPU caches. The largely depends on the</div> <div class="ContentPasted0">> @@ -426,7 +338,8 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * i.e @cache_coherent is zero.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * On platforms that share the LLC with the CPU(HAS_LLC), all GT memory</div> <div class="ContentPasted0">> - * access will automatically snoop the CPU caches(even with CACHE_NONE).</div> <div class="ContentPasted0">> + * access will automatically snoop the CPU caches (even with</div> <div class="ContentPasted0">> + * I915_CACHE(UC)).</div> <div class="ContentPasted0">> * The one exception is when dealing with the display engine, like with</div> <div class="ContentPasted0">> * scanout surfaces. To handle this the kernel will always flush the</div> <div class="ContentPasted0">> * surface out of the CPU caches when preparing it for scanout. Also</div> <div class="ContentPasted0">> @@ -437,8 +350,8 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * Something strange here is why @cache_coherent is not a simple</div> <div class="ContentPasted0">> * boolean, i.e coherent vs non-coherent. The reasoning for this is back</div> <div class="ContentPasted0">> * to the display engine not being fully coherent. As a result scanout</div> <div class="ContentPasted0">> - * surfaces will either be marked as I915_CACHE_NONE or I915_CACHE_WT.</div> <div class="ContentPasted0">> - * In the case of seeing I915_CACHE_NONE the kernel makes the assumption</div> <div class="ContentPasted0">> + * surfaces will either be marked as I915_CACHE(UC) or I915_CACHE(WT).</div> <div class="ContentPasted0">> + * In the case of seeing I915_CACHE(UC) the kernel makes the assumption</div> <div class="ContentPasted0">> * that this is likely a scanout surface, and will set @cache_coherent</div> <div class="ContentPasted0">> * as only I915_BO_CACHE_COHERENT_FOR_READ, on platforms with the shared</div> <div class="ContentPasted0">> * LLC. The kernel uses this to always flush writes through the CPU</div> <div class="ContentPasted0">> @@ -447,7 +360,7 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * flushing the surface just before doing the scanout. This does mean</div> <div class="ContentPasted0">> * we might unnecessarily flush non-scanout objects in some places, but</div> <div class="ContentPasted0">> * the default assumption is that all normal objects should be using</div> <div class="ContentPasted0">> - * I915_CACHE_LLC, at least on platforms with the shared LLC.</div> <div class="ContentPasted0">> + * I915_CACHE(WB), at least on platforms with the shared LLC.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * Supported values:</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> @@ -459,13 +372,13 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * optimisation we try to flush any writes as early as possible to avoid</div> <div class="ContentPasted0">> * stalling later.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> - * Thus for scanout surfaces using I915_CACHE_NONE, on shared LLC</div> <div class="ContentPasted0">> + * Thus for scanout surfaces using I915_CACHE(UC), on shared LLC</div> <div class="ContentPasted0">> * platforms, we use:</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * While for normal objects that are fully coherent, including special</div> <div class="ContentPasted0">> - * scanout surfaces marked as I915_CACHE_WT, we use:</div> <div class="ContentPasted0">> + * scanout surfaces marked as I915_CACHE(WT), we use:</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ |</div> <div class="ContentPasted0">> * I915_BO_CACHE_COHERENT_FOR_WRITE</div> <div class="ContentPasted0">> @@ -492,9 +405,7 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * Check for @pat_set_by_user to find out if an object has pat index set</div> <div class="ContentPasted0">> * by userspace. The ioctl's to change cache settings have also been</div> <div class="ContentPasted0">> * disabled for the objects with pat_index set by userspace. Please don't</div> <div class="ContentPasted0">> - * assume @cache_dirty is set as describe here. Also see helper function</div> <div class="ContentPasted0">> - * i915_gem_object_has_cache_level() for possible ways to bypass the use</div> <div class="ContentPasted0">> - * of this field.</div> <div class="ContentPasted0">> + * assume @cache_dirty is set as describe here.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * Track if we are we dirty with writes through the CPU cache for this</div> <div class="ContentPasted0">> * object. As a result reading directly from main memory might yield</div> <div class="ContentPasted0">> @@ -517,7 +428,7 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * for both reads and writes though the CPU cache.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * Note that on shared LLC platforms we still apply the heavy flush for</div> <div class="ContentPasted0">> - * I915_CACHE_NONE objects, under the assumption that this is going to</div> <div class="ContentPasted0">> + * I915_CACHE(UC) objects, under the assumption that this is going to</div> <div class="ContentPasted0">> * be used for scanout.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * Update: On some hardware there is now also the 'Bypass LLC' MOCS</div> <div class="ContentPasted0">> @@ -530,15 +441,15 @@ struct drm_i915_gem_object {</div> <div class="ContentPasted0">> * information leak. The driver currently ensures:</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * 1. All userspace objects, by default, have @cache_level set as</div> <div class="ContentPasted0">> - * I915_CACHE_NONE. The only exception is userptr objects, where we</div> <div class="ContentPasted0">> - * instead force I915_CACHE_LLC, but we also don't allow userspace to</div> <div class="ContentPasted0">> + * I915_CACHE(UC). The only exception is userptr objects, where we</div> <div class="ContentPasted0">> + * instead force I915_CACHE(WB), but we also don't allow userspace to</div> <div class="ContentPasted0">> * ever change the @cache_level for such objects. Another special case</div> <div class="ContentPasted0">> * is dma-buf, which doesn't rely on @cache_dirty, but there we</div> <div class="ContentPasted0">> * always do a forced flush when acquiring the pages, if there is a</div> <div class="ContentPasted0">> * chance that the pages can be read directly from main memory with</div> <div class="ContentPasted0">> * the GPU.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> - * 2. All I915_CACHE_NONE objects have @cache_dirty initially true.</div> <div class="ContentPasted0">> + * 2. All I915_CACHE(UC) objects have @cache_dirty initially true.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> * 3. All swapped-out objects(i.e shmem) have @cache_dirty set to</div> <div class="ContentPasted0">> * true.</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c</div> <div class="ContentPasted0">> index 33d5d5178103..538b3029de35 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c</div> <div class="ContentPasted0">> @@ -375,7 +375,7 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> * On non-LLC igfx platforms, force the flush-on-acquire if this is ever</div> <div class="ContentPasted0">> * swapped-in. Our async flush path is not trust worthy enough yet(and</div> <div class="ContentPasted0">> * happens in the wrong order), and with some tricks it's conceivable</div> <div class="ContentPasted0">> - * for userspace to change the cache-level to I915_CACHE_NONE after the</div> <div class="ContentPasted0">> + * for userspace to change the cache-level to I915_CACHE(UC) after the</div> <div class="ContentPasted0">> * pages are swapped-in, and since execbuf binds the object before doing</div> <div class="ContentPasted0">> * the async flush, we have a race window.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> @@ -577,7 +577,7 @@ static int shmem_object_init(struct intel_memory_region *mem,</div> <div class="ContentPasted0">> static struct lock_class_key lock_class;</div> <div class="ContentPasted0">> struct drm_i915_private *i915 = mem->i915;</div> <div class="ContentPasted0">> struct address_space *mapping;</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> + i915_cache_t cache;</div> <div class="ContentPasted0">> gfp_t mask;</div> <div class="ContentPasted0">> int ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -621,11 +621,11 @@ static int shmem_object_init(struct intel_memory_region *mem,</div> <div class="ContentPasted0">> * However, we maintain the display planes as UC, and so</div> <div class="ContentPasted0">> * need to rebind when first used as such.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - cache_level = I915_CACHE_LLC;</div> <div class="ContentPasted0">> + cache = I915_CACHE(WB);</div> <div class="ContentPasted0">> else</div> <div class="ContentPasted0">> - cache_level = I915_CACHE_NONE;</div> <div class="ContentPasted0">> + cache = I915_CACHE(UC);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, cache);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> i915_gem_object_init_memory_region(obj, mem);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c</div> <div class="ContentPasted0">> index 3b094d36a0b0..e3a3b824a536 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c</div> <div class="ContentPasted0">> @@ -563,11 +563,8 @@ static void dbg_poison(struct i915_ggtt *ggtt,</div> <div class="ContentPasted0">> while (size) {</div> <div class="ContentPasted0">> void __iomem *s;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ggtt->vm.insert_page(&ggtt->vm, addr,</div> <div class="ContentPasted0">> - ggtt->error_capture.start,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(ggtt->vm.i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + ggtt->vm.insert_page(&ggtt->vm, addr, ggtt->error_capture.start,</div> <div class="ContentPasted0">> + ggtt->vm.i915->pat_uc, 0);</div> <div class="ContentPasted0">> mb();</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> s = io_mapping_map_wc(&ggtt->iomap,</div> <div class="ContentPasted0">> @@ -679,7 +676,6 @@ static int __i915_gem_object_create_stolen(struct intel_memory_region *mem,</div> <div class="ContentPasted0">> struct drm_mm_node *stolen)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> static struct lock_class_key lock_class;</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> unsigned int flags;</div> <div class="ContentPasted0">> int err;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -694,8 +690,9 @@ static int __i915_gem_object_create_stolen(struct intel_memory_region *mem,</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->stolen = stolen;</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT;</div> <div class="ContentPasted0">> - cache_level = HAS_LLC(mem->i915) ? I915_CACHE_LLC : I915_CACHE_NONE;</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, HAS_LLC(mem->i915) ?</div> <div class="ContentPasted0">> + I915_CACHE(WB) :</div> <div class="ContentPasted0">> + I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (WARN_ON(!i915_gem_object_trylock(obj, NULL)))</div> <div class="ContentPasted0">> return -EBUSY;</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c</div> <div class="ContentPasted0">> index 7078af2f8f79..52c0273001b2 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c</div> <div class="ContentPasted0">> @@ -48,14 +48,23 @@ void i915_ttm_migrate_set_ban_memcpy(bool ban)</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> #endif</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -static enum i915_cache_level</div> <div class="ContentPasted0">> -i915_ttm_cache_level(struct drm_i915_private *i915, struct ttm_resource *res,</div> <div class="ContentPasted0">> +static i915_cache_t</div> <div class="ContentPasted0">> +i915_ttm_cache(struct drm_i915_private *i915, struct ttm_resource *res,</div> <div class="ContentPasted0">> + struct ttm_tt *ttm)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + return ((HAS_LLC(i915) || HAS_SNOOP(i915)) &&</div> <div class="ContentPasted0">> + !i915_ttm_gtt_binds_lmem(res) &&</div> <div class="ContentPasted0">> + ttm->caching == ttm_cached) ? I915_CACHE(WB) : I915_CACHE(UC);</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +static unsigned int</div> <div class="ContentPasted0">> +i915_ttm_cache_pat(struct drm_i915_private *i915, struct ttm_resource *res,</div> <div class="ContentPasted0">> struct ttm_tt *ttm)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> return ((HAS_LLC(i915) || HAS_SNOOP(i915)) &&</div> <div class="ContentPasted0">> !i915_ttm_gtt_binds_lmem(res) &&</div> <div class="ContentPasted0">> - ttm->caching == ttm_cached) ? I915_CACHE_LLC :</div> <div class="ContentPasted0">> - I915_CACHE_NONE;</div> <div class="ContentPasted0">> + ttm->caching == ttm_cached) ? i915->pat_wb :</div> <div class="ContentPasted0">> + i915->pat_uc;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static struct intel_memory_region *</div> <div class="ContentPasted0">> @@ -102,8 +111,8 @@ void i915_ttm_adjust_domains_after_move(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> unsigned int mem_flags;</div> <div class="ContentPasted0">> + i915_cache_t cache;</div> <div class="ContentPasted0">> unsigned int i;</div> <div class="ContentPasted0">> int mem_type;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -116,13 +125,13 @@ void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> if (!bo->resource) {</div> <div class="ContentPasted0">> mem_flags = I915_BO_FLAG_STRUCT_PAGE;</div> <div class="ContentPasted0">> mem_type = I915_PL_SYSTEM;</div> <div class="ContentPasted0">> - cache_level = I915_CACHE_NONE;</div> <div class="ContentPasted0">> + cache = I915_CACHE(UC);</div> <div class="ContentPasted0">> } else {</div> <div class="ContentPasted0">> mem_flags = i915_ttm_cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM :</div> <div class="ContentPasted0">> I915_BO_FLAG_STRUCT_PAGE;</div> <div class="ContentPasted0">> mem_type = bo->resource->mem_type;</div> <div class="ContentPasted0">> - cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), bo->resource,</div> <div class="ContentPasted0">> - bo->ttm);</div> <div class="ContentPasted0">> + cache = i915_ttm_cache(to_i915(bo->base.dev), bo->resource,</div> <div class="ContentPasted0">> + bo->ttm);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /*</div> <div class="ContentPasted0">> @@ -147,7 +156,7 @@ void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> obj->mem_flags &= ~(I915_BO_FLAG_STRUCT_PAGE | I915_BO_FLAG_IOMEM);</div> <div class="ContentPasted0">> obj->mem_flags |= mem_flags;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, cache);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /**</div> <div class="ContentPasted0">> @@ -196,7 +205,7 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo,</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);</div> <div class="ContentPasted0">> struct i915_request *rq;</div> <div class="ContentPasted0">> struct ttm_tt *src_ttm = bo->ttm;</div> <div class="ContentPasted0">> - enum i915_cache_level src_level, dst_level;</div> <div class="ContentPasted0">> + unsigned int src_pat, dst_pat;</div> <div class="ContentPasted0">> int ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (!to_gt(i915)->migrate.context || intel_gt_is_wedged(to_gt(i915)))</div> <div class="ContentPasted0">> @@ -206,16 +215,15 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo,</div> <div class="ContentPasted0">> if (I915_SELFTEST_ONLY(fail_gpu_migration))</div> <div class="ContentPasted0">> clear = true;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - dst_level = i915_ttm_cache_level(i915, dst_mem, dst_ttm);</div> <div class="ContentPasted0">> + dst_pat = i915_ttm_cache_pat(i915, dst_mem, dst_ttm);</div> <div class="ContentPasted0">> if (clear) {</div> <div class="ContentPasted0">> if (bo->type == ttm_bo_type_kernel &&</div> <div class="ContentPasted0">> !I915_SELFTEST_ONLY(fail_gpu_migration))</div> <div class="ContentPasted0">> return ERR_PTR(-EINVAL);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> intel_engine_pm_get(to_gt(i915)->migrate.context->engine);</div> <div class="ContentPasted0">> - ret = intel_context_migrate_clear(to_gt(i915)->migrate.context, deps,</div> <div class="ContentPasted0">> - dst_st->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915, dst_level),</div> <div class="ContentPasted0">> + ret = intel_context_migrate_clear(to_gt(i915)->migrate.context,</div> <div class="ContentPasted0">> + deps, dst_st->sgl, dst_pat,</div> <div class="ContentPasted0">> i915_ttm_gtt_binds_lmem(dst_mem),</div> <div class="ContentPasted0">> 0, &rq);</div> <div class="ContentPasted0">> } else {</div> <div class="ContentPasted0">> @@ -225,14 +233,13 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo,</div> <div class="ContentPasted0">> if (IS_ERR(src_rsgt))</div> <div class="ContentPasted0">> return ERR_CAST(src_rsgt);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - src_level = i915_ttm_cache_level(i915, bo->resource, src_ttm);</div> <div class="ContentPasted0">> + src_pat = i915_ttm_cache_pat(i915, bo->resource, src_ttm);</div> <div class="ContentPasted0">> intel_engine_pm_get(to_gt(i915)->migrate.context->engine);</div> <div class="ContentPasted0">> ret = intel_context_migrate_copy(to_gt(i915)->migrate.context,</div> <div class="ContentPasted0">> deps, src_rsgt->table.sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915, src_level),</div> <div class="ContentPasted0">> + src_pat,</div> <div class="ContentPasted0">> i915_ttm_gtt_binds_lmem(bo->resource),</div> <div class="ContentPasted0">> - dst_st->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915, dst_level),</div> <div class="ContentPasted0">> + dst_st->sgl, dst_pat,</div> <div class="ContentPasted0">> i915_ttm_gtt_binds_lmem(dst_mem),</div> <div class="ContentPasted0">> &rq);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c</div> <div class="ContentPasted0">> index 1d3ebdf4069b..7d50d43dfbfa 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c</div> <div class="ContentPasted0">> @@ -553,7 +553,7 @@ i915_gem_userptr_ioctl(struct drm_device *dev,</div> <div class="ContentPasted0">> obj->mem_flags = I915_BO_FLAG_STRUCT_PAGE;</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->userptr.ptr = args->user_ptr;</div> <div class="ContentPasted0">> obj->userptr.notifier_seq = ULONG_MAX;</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c b/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c</div> <div class="ContentPasted0">> index bac957755068..ad7798d34c74 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c</div> <div class="ContentPasted0">> @@ -104,7 +104,6 @@ huge_gem_object(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> static struct lock_class_key lock_class;</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> GEM_BUG_ON(!phys_size || phys_size > dma_size);</div> <div class="ContentPasted0">> GEM_BUG_ON(!IS_ALIGNED(phys_size, PAGE_SIZE));</div> <div class="ContentPasted0">> @@ -123,8 +122,9 @@ huge_gem_object(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> - cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, HAS_LLC(i915) ?</div> <div class="ContentPasted0">> + I915_CACHE(WB) :</div> <div class="ContentPasted0">> + I915_CACHE(UC));</div> <div class="ContentPasted0">> obj->scratch = phys_size;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return obj;</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c</div> <div class="ContentPasted0">> index df6c9a84252c..060830c51c2f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c</div> <div class="ContentPasted0">> @@ -177,7 +177,6 @@ huge_pages_object(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> static struct lock_class_key lock_class;</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> - unsigned int cache_level;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> GEM_BUG_ON(!size);</div> <div class="ContentPasted0">> GEM_BUG_ON(!IS_ALIGNED(size, BIT(__ffs(page_mask))));</div> <div class="ContentPasted0">> @@ -200,8 +199,9 @@ huge_pages_object(struct drm_i915_private *i915,</div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, cache_level);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, HAS_LLC(i915) ?</div> <div class="ContentPasted0">> + I915_CACHE(WB) :</div> <div class="ContentPasted0">> + I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->mm.page_mask = page_mask;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -354,7 +354,7 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> - obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return obj;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c</div> <div class="ContentPasted0">> index c2bdc133c89a..fb69f667652a 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c</div> <div class="ContentPasted0">> @@ -226,9 +226,7 @@ static int gen6_ppgtt_init_scratch(struct gen6_ppgtt *ppgtt)</div> <div class="ContentPasted0">> return ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vm->scratch[0]->encode =</div> <div class="ContentPasted0">> - vm->pte_encode(px_dma(vm->scratch[0]),</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + vm->pte_encode(px_dma(vm->scratch[0]), vm->i915->pat_uc,</div> <div class="ContentPasted0">> PTE_READ_ONLY);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vm->scratch[1] = vm->alloc_pt_dma(vm, I915_GTT_PAGE_SIZE_4K);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c</div> <div class="ContentPasted0">> index f948d33e5ec5..bd5a7d295251 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c</div> <div class="ContentPasted0">> @@ -16,11 +16,11 @@</div> <div class="ContentPasted0">> #include "intel_gtt.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static u64 gen8_pde_encode(const dma_addr_t addr,</div> <div class="ContentPasted0">> - const enum i915_cache_level level)</div> <div class="ContentPasted0">> + const enum i915_cache_mode cache_mode)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> u64 pde = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (level != I915_CACHE_NONE)</div> <div class="ContentPasted0">> + if (cache_mode != I915_CACHE_MODE_UC)</div> <div class="ContentPasted0">> pde |= PPAT_CACHED_PDE;</div> <div class="ContentPasted0">> else</div> <div class="ContentPasted0">> pde |= PPAT_UNCACHED;</div> <div class="ContentPasted0">> @@ -40,16 +40,11 @@ static u64 gen8_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> if (flags & PTE_LM)</div> <div class="ContentPasted0">> pte |= GEN12_PPGTT_PTE_LM;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * For pre-gen12 platforms pat_index is the same as enum</div> <div class="ContentPasted0">> - * i915_cache_level, so the switch-case here is still valid.</div> <div class="ContentPasted0">> - * See translation table defined by LEGACY_CACHELEVEL.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> switch (pat_index) {</div> <div class="ContentPasted0">> - case I915_CACHE_NONE:</div> <div class="ContentPasted0">> + case 0:</div> <div class="ContentPasted0">> pte |= PPAT_UNCACHED;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> - case I915_CACHE_WT:</div> <div class="ContentPasted0">> + case 3:</div> <div class="ContentPasted0">> pte |= PPAT_DISPLAY_ELLC;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> default:</div> <div class="ContentPasted0">> @@ -853,9 +848,7 @@ static int gen8_init_scratch(struct i915_address_space *vm)</div> <div class="ContentPasted0">> pte_flags |= PTE_LM;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vm->scratch[0]->encode =</div> <div class="ContentPasted0">> - vm->pte_encode(px_dma(vm->scratch[0]),</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + vm->pte_encode(px_dma(vm->scratch[0]), vm->i915->pat_uc,</div> <div class="ContentPasted0">> pte_flags);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> for (i = 1; i <= vm->top; i++) {</div> <div class="ContentPasted0">> @@ -874,7 +867,7 @@ static int gen8_init_scratch(struct i915_address_space *vm)</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> fill_px(obj, vm->scratch[i - 1]->encode);</div> <div class="ContentPasted0">> - obj->encode = gen8_pde_encode(px_dma(obj), I915_CACHE_NONE);</div> <div class="ContentPasted0">> + obj->encode = gen8_pde_encode(px_dma(obj), I915_CACHE_MODE_UC);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vm->scratch[i] = obj;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c</div> <div class="ContentPasted0">> index 0aff5bb13c53..b1e846682cd9 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c</div> <div class="ContentPasted0">> @@ -1103,7 +1103,7 @@ static int init_status_page(struct intel_engine_cs *engine)</div> <div class="ContentPasted0">> return PTR_ERR(obj);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);</div> <div class="ContentPasted0">> if (IS_ERR(vma)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c</div> <div class="ContentPasted0">> index dd0ed941441a..05fafb076556 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c</div> <div class="ContentPasted0">> @@ -921,9 +921,7 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)</div> <div class="ContentPasted0">> pte_flags |= PTE_LM;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> ggtt->vm.scratch[0]->encode =</div> <div class="ContentPasted0">> - ggtt->vm.pte_encode(px_dma(ggtt->vm.scratch[0]),</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + ggtt->vm.pte_encode(px_dma(ggtt->vm.scratch[0]), i915->pat_uc,</div> <div class="ContentPasted0">> pte_flags);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return 0;</div> <div class="ContentPasted0">> @@ -1013,11 +1011,6 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)</div> <div class="ContentPasted0">> return ggtt_probe_common(ggtt, size);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -/*</div> <div class="ContentPasted0">> - * For pre-gen8 platforms pat_index is the same as enum i915_cache_level,</div> <div class="ContentPasted0">> - * so the switch-case statements in these PTE encode functions are still valid.</div> <div class="ContentPasted0">> - * See translation table LEGACY_CACHELEVEL.</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> static u64 snb_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> unsigned int pat_index,</div> <div class="ContentPasted0">> u32 flags)</div> <div class="ContentPasted0">> @@ -1025,11 +1018,11 @@ static u64 snb_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> gen6_pte_t pte = GEN6_PTE_ADDR_ENCODE(addr) | GEN6_PTE_VALID;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> switch (pat_index) {</div> <div class="ContentPasted0">> - case I915_CACHE_L3_LLC:</div> <div class="ContentPasted0">> - case I915_CACHE_LLC:</div> <div class="ContentPasted0">> + case 1:</div> <div class="ContentPasted0">> + case 2:</div> <div class="ContentPasted0">> pte |= GEN6_PTE_CACHE_LLC;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> - case I915_CACHE_NONE:</div> <div class="ContentPasted0">> + case 0:</div> <div class="ContentPasted0">> pte |= GEN6_PTE_UNCACHED;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> default:</div> <div class="ContentPasted0">> @@ -1046,13 +1039,13 @@ static u64 ivb_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> gen6_pte_t pte = GEN6_PTE_ADDR_ENCODE(addr) | GEN6_PTE_VALID;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> switch (pat_index) {</div> <div class="ContentPasted0">> - case I915_CACHE_L3_LLC:</div> <div class="ContentPasted0">> + case 2:</div> <div class="ContentPasted0">> pte |= GEN7_PTE_CACHE_L3_LLC;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> - case I915_CACHE_LLC:</div> <div class="ContentPasted0">> + case 1:</div> <div class="ContentPasted0">> pte |= GEN6_PTE_CACHE_LLC;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> - case I915_CACHE_NONE:</div> <div class="ContentPasted0">> + case 0:</div> <div class="ContentPasted0">> pte |= GEN6_PTE_UNCACHED;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> default:</div> <div class="ContentPasted0">> @@ -1071,7 +1064,7 @@ static u64 byt_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> if (!(flags & PTE_READ_ONLY))</div> <div class="ContentPasted0">> pte |= BYT_PTE_WRITEABLE;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (pat_index != I915_CACHE_NONE)</div> <div class="ContentPasted0">> + if (pat_index)</div> <div class="ContentPasted0">> pte |= BYT_PTE_SNOOPED_BY_CPU_CACHES;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return pte;</div> <div class="ContentPasted0">> @@ -1083,7 +1076,7 @@ static u64 hsw_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> gen6_pte_t pte = HSW_PTE_ADDR_ENCODE(addr) | GEN6_PTE_VALID;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - if (pat_index != I915_CACHE_NONE)</div> <div class="ContentPasted0">> + if (pat_index)</div> <div class="ContentPasted0">> pte |= HSW_WB_LLC_AGE3;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> return pte;</div> <div class="ContentPasted0">> @@ -1096,9 +1089,9 @@ static u64 iris_pte_encode(dma_addr_t addr,</div> <div class="ContentPasted0">> gen6_pte_t pte = HSW_PTE_ADDR_ENCODE(addr) | GEN6_PTE_VALID;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> switch (pat_index) {</div> <div class="ContentPasted0">> - case I915_CACHE_NONE:</div> <div class="ContentPasted0">> + case 0:</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> - case I915_CACHE_WT:</div> <div class="ContentPasted0">> + case 3:</div> <div class="ContentPasted0">> pte |= HSW_WT_ELLC_LLC_AGE3;</div> <div class="ContentPasted0">> break;</div> <div class="ContentPasted0">> default:</div> <div class="ContentPasted0">> @@ -1298,9 +1291,7 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm)</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> vma->resource->bound_flags = 0;</div> <div class="ContentPasted0">> vma->ops->bind_vma(vm, NULL, vma->resource,</div> <div class="ContentPasted0">> - obj ? obj->pat_index :</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + obj ? obj->pat_index : vm->i915->pat_uc,</div> <div class="ContentPasted0">> was_bound);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (obj) { /* only used during resume => exclusive access */</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c b/drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c</div> <div class="ContentPasted0">> index 866c416afb73..fedf02c9ecb6 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c</div> <div class="ContentPasted0">> @@ -21,7 +21,7 @@ static void gmch_ggtt_insert_page(struct i915_address_space *vm,</div> <div class="ContentPasted0">> unsigned int pat_index,</div> <div class="ContentPasted0">> u32 unused)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> - unsigned int flags = (pat_index == I915_CACHE_NONE) ?</div> <div class="ContentPasted0">> + unsigned int flags = (pat_index == 0) ?</div> <div class="ContentPasted0">> AGP_USER_MEMORY : AGP_USER_CACHED_MEMORY;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> intel_gmch_gtt_insert_page(addr, offset >> PAGE_SHIFT, flags);</div> <div class="ContentPasted0">> @@ -32,7 +32,7 @@ static void gmch_ggtt_insert_entries(struct i915_address_space *vm,</div> <div class="ContentPasted0">> unsigned int pat_index,</div> <div class="ContentPasted0">> u32 unused)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> - unsigned int flags = (pat_index == I915_CACHE_NONE) ?</div> <div class="ContentPasted0">> + unsigned int flags = (pat_index == 0) ?</div> <div class="ContentPasted0">> AGP_USER_MEMORY : AGP_USER_CACHED_MEMORY;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> intel_gmch_gtt_insert_sg_entries(vma_res->bi.pages, vma_res->start >> PAGE_SHIFT,</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.c b/drivers/gpu/drm/i915/gt/intel_gtt.c</div> <div class="ContentPasted0">> index 126269a0d728..8729eb244d28 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_gtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_gtt.c</div> <div class="ContentPasted0">> @@ -676,7 +676,7 @@ __vm_create_scratch_for_read(struct i915_address_space *vm, unsigned long size)</div> <div class="ContentPasted0">> if (IS_ERR(obj))</div> <div class="ContentPasted0">> return ERR_CAST(obj);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHING_CACHED);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_vma_instance(obj, vm, NULL);</div> <div class="ContentPasted0">> if (IS_ERR(vma)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h</div> <div class="ContentPasted0">> index 7192a534a654..af4277c1d577 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_gtt.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h</div> <div class="ContentPasted0">> @@ -636,7 +636,8 @@ void</div> <div class="ContentPasted0">> __set_pd_entry(struct i915_page_directory * const pd,</div> <div class="ContentPasted0">> const unsigned short idx,</div> <div class="ContentPasted0">> struct i915_page_table *pt,</div> <div class="ContentPasted0">> - u64 (*encode)(const dma_addr_t, const enum i915_cache_level));</div> <div class="ContentPasted0">> + u64 (*encode)(const dma_addr_t,</div> <div class="ContentPasted0">> + const enum i915_cache_mode cache_mode));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define set_pd_entry(pd, idx, to) \</div> <div class="ContentPasted0">> __set_pd_entry((pd), (idx), px_pt(to), gen8_pde_encode)</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c</div> <div class="ContentPasted0">> index 6023288b0e2d..81f7834cc2db 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_migrate.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_migrate.c</div> <div class="ContentPasted0">> @@ -45,9 +45,7 @@ static void xehpsdv_toggle_pdes(struct i915_address_space *vm,</div> <div class="ContentPasted0">> * Insert a dummy PTE into every PT that will map to LMEM to ensure</div> <div class="ContentPasted0">> * we have a correctly setup PDE structure for later use.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - vm->insert_page(vm, 0, d->offset,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE),</div> <div class="ContentPasted0">> - PTE_LM);</div> <div class="ContentPasted0">> + vm->insert_page(vm, 0, d->offset, vm->i915->pat_uc, PTE_LM);</div> <div class="ContentPasted0">> GEM_BUG_ON(!pt->is_compact);</div> <div class="ContentPasted0">> d->offset += SZ_2M;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> @@ -65,9 +63,7 @@ static void xehpsdv_insert_pte(struct i915_address_space *vm,</div> <div class="ContentPasted0">> * alignment is 64K underneath for the pt, and we are careful</div> <div class="ContentPasted0">> * not to access the space in the void.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - vm->insert_page(vm, px_dma(pt), d->offset,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE),</div> <div class="ContentPasted0">> - PTE_LM);</div> <div class="ContentPasted0">> + vm->insert_page(vm, px_dma(pt), d->offset, vm->i915->pat_uc, PTE_LM);</div> <div class="ContentPasted0">> d->offset += SZ_64K;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -77,8 +73,7 @@ static void insert_pte(struct i915_address_space *vm,</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> struct insert_pte_data *d = data;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - vm->insert_page(vm, px_dma(pt), d->offset,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE),</div> <div class="ContentPasted0">> + vm->insert_page(vm, px_dma(pt), d->offset, vm->i915->pat_uc,</div> <div class="ContentPasted0">> i915_gem_object_is_lmem(pt->base) ? PTE_LM : 0);</div> <div class="ContentPasted0">> d->offset += PAGE_SIZE;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/gt/intel_ppgtt.c</div> <div class="ContentPasted0">> index 436756bfbb1a..3e461d4f3693 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c</div> <div class="ContentPasted0">> @@ -98,14 +98,16 @@ void</div> <div class="ContentPasted0">> __set_pd_entry(struct i915_page_directory * const pd,</div> <div class="ContentPasted0">> const unsigned short idx,</div> <div class="ContentPasted0">> struct i915_page_table * const to,</div> <div class="ContentPasted0">> - u64 (*encode)(const dma_addr_t, const enum i915_cache_level))</div> <div class="ContentPasted0">> + u64 (*encode)(const dma_addr_t,</div> <div class="ContentPasted0">> + const enum i915_cache_mode cache_mode))</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> /* Each thread pre-pins the pd, and we may have a thread per pde. */</div> <div class="ContentPasted0">> GEM_BUG_ON(atomic_read(px_used(pd)) > NALLOC * I915_PDES);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> atomic_inc(px_used(pd));</div> <div class="ContentPasted0">> pd->entry[idx] = to;</div> <div class="ContentPasted0">> - write_dma_entry(px_base(pd), idx, encode(px_dma(to), I915_CACHE_LLC));</div> <div class="ContentPasted0">> + write_dma_entry(px_base(pd), idx,</div> <div class="ContentPasted0">> + encode(px_dma(to), I915_CACHE_MODE_WB));</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> void</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c</div> <div class="ContentPasted0">> index 3fd795c3263f..fc7781b47bbf 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c</div> <div class="ContentPasted0">> @@ -551,7 +551,7 @@ alloc_context_vma(struct intel_engine_cs *engine)</div> <div class="ContentPasted0">> * later platforms don't have L3 control bits in the PTE.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> if (IS_IVYBRIDGE(i915))</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_L3_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, _I915_CACHE(WB, L3));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);</div> <div class="ContentPasted0">> if (IS_ERR(vma)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.c b/drivers/gpu/drm/i915/gt/intel_timeline.c</div> <div class="ContentPasted0">> index b9640212d659..ceb2616fb8d9 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/intel_timeline.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/intel_timeline.c</div> <div class="ContentPasted0">> @@ -26,7 +26,7 @@ static struct i915_vma *hwsp_alloc(struct intel_gt *gt)</div> <div class="ContentPasted0">> if (IS_ERR(obj))</div> <div class="ContentPasted0">> return ERR_CAST(obj);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);</div> <div class="ContentPasted0">> if (IS_ERR(vma))</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c</div> <div class="ContentPasted0">> index 8b0d84f2aad2..f0baf86b6848 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c</div> <div class="ContentPasted0">> @@ -64,7 +64,7 @@ static int hang_init(struct hang *h, struct intel_gt *gt)</div> <div class="ContentPasted0">> goto err_hws;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(h->hws, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(h->hws, I915_CACHE(WB));</div> <div class="ContentPasted0">> vaddr = i915_gem_object_pin_map_unlocked(h->hws, I915_MAP_WB);</div> <div class="ContentPasted0">> if (IS_ERR(vaddr)) {</div> <div class="ContentPasted0">> err = PTR_ERR(vaddr);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c</div> <div class="ContentPasted0">> index 3def5ca72dec..a67ede65d816 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_migrate.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c</div> <div class="ContentPasted0">> @@ -904,8 +904,7 @@ static int perf_clear_blt(void *arg)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> err = __perf_clear_blt(gt->migrate.context,</div> <div class="ContentPasted0">> dst->mm.pages->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + gt->i915->pat_uc,</div> <div class="ContentPasted0">> i915_gem_object_is_lmem(dst),</div> <div class="ContentPasted0">> sizes[i]);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -995,12 +994,10 @@ static int perf_copy_blt(void *arg)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> err = __perf_copy_blt(gt->migrate.context,</div> <div class="ContentPasted0">> src->mm.pages->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + gt->i915->pat_uc,</div> <div class="ContentPasted0">> i915_gem_object_is_lmem(src),</div> <div class="ContentPasted0">> dst->mm.pages->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + gt->i915->pat_uc,</div> <div class="ContentPasted0">> i915_gem_object_is_lmem(dst),</div> <div class="ContentPasted0">> sz);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c</div> <div class="ContentPasted0">> index 79aa6ac66ad2..327dc9294e0f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_reset.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c</div> <div class="ContentPasted0">> @@ -84,11 +84,8 @@ __igt_reset_stolen(struct intel_gt *gt,</div> <div class="ContentPasted0">> void __iomem *s;</div> <div class="ContentPasted0">> void *in;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ggtt->vm.insert_page(&ggtt->vm, dma,</div> <div class="ContentPasted0">> - ggtt->error_capture.start,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start,</div> <div class="ContentPasted0">> + gt->i915->pat_uc, 0);</div> <div class="ContentPasted0">> mb();</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> s = io_mapping_map_wc(&ggtt->iomap,</div> <div class="ContentPasted0">> @@ -127,11 +124,8 @@ __igt_reset_stolen(struct intel_gt *gt,</div> <div class="ContentPasted0">> void *in;</div> <div class="ContentPasted0">> u32 x;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ggtt->vm.insert_page(&ggtt->vm, dma,</div> <div class="ContentPasted0">> - ggtt->error_capture.start,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start,</div> <div class="ContentPasted0">> + gt->i915->pat_uc, 0);</div> <div class="ContentPasted0">> mb();</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> s = io_mapping_map_wc(&ggtt->iomap,</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c</div> <div class="ContentPasted0">> index 39c3ec12df1a..87299158815f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_timeline.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c</div> <div class="ContentPasted0">> @@ -837,6 +837,7 @@ static int setup_watcher(struct hwsp_watcher *w, struct intel_gt *gt,</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* keep the same cache settings as timeline */</div> <div class="ContentPasted0">> i915_gem_object_set_pat_index(obj, tl->hwsp_ggtt->obj->pat_index);</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> w->map = i915_gem_object_pin_map_unlocked(obj,</div> <div class="ContentPasted0">> page_unmask_bits(tl->hwsp_ggtt->obj->mm.mapping));</div> <div class="ContentPasted0">> if (IS_ERR(w->map)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_tlb.c b/drivers/gpu/drm/i915/gt/selftest_tlb.c</div> <div class="ContentPasted0">> index 3bd6b540257b..6049f01be219 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_tlb.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_tlb.c</div> <div class="ContentPasted0">> @@ -36,8 +36,6 @@ pte_tlbinv(struct intel_context *ce,</div> <div class="ContentPasted0">> u64 length,</div> <div class="ContentPasted0">> struct rnd_state *prng)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> - const unsigned int pat_index =</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(ce->vm->i915, I915_CACHE_NONE);</div> <div class="ContentPasted0">> struct drm_i915_gem_object *batch;</div> <div class="ContentPasted0">> struct drm_mm_node vb_node;</div> <div class="ContentPasted0">> struct i915_request *rq;</div> <div class="ContentPasted0">> @@ -157,7 +155,8 @@ pte_tlbinv(struct intel_context *ce,</div> <div class="ContentPasted0">> /* Flip the PTE between A and B */</div> <div class="ContentPasted0">> if (i915_gem_object_is_lmem(vb->obj))</div> <div class="ContentPasted0">> pte_flags |= PTE_LM;</div> <div class="ContentPasted0">> - ce->vm->insert_entries(ce->vm, &vb_res, pat_index, pte_flags);</div> <div class="ContentPasted0">> + ce->vm->insert_entries(ce->vm, &vb_res, ce->vm->i915->pat_uc,</div> <div class="ContentPasted0">> + pte_flags);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* Flush the PTE update to concurrent HW */</div> <div class="ContentPasted0">> tlbinv(ce->vm, addr & -length, length);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/selftest_workarounds.c b/drivers/gpu/drm/i915/gt/selftest_workarounds.c</div> <div class="ContentPasted0">> index 14a8b25b6204..900b4e19087f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/selftest_workarounds.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/selftest_workarounds.c</div> <div class="ContentPasted0">> @@ -111,7 +111,7 @@ read_nonprivs(struct intel_context *ce)</div> <div class="ContentPasted0">> if (IS_ERR(result))</div> <div class="ContentPasted0">> return result;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(result, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(result, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> cs = i915_gem_object_pin_map_unlocked(result, I915_MAP_WB);</div> <div class="ContentPasted0">> if (IS_ERR(cs)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c</div> <div class="ContentPasted0">> index 2eb891b270ae..3a862051423c 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c</div> <div class="ContentPasted0">> @@ -749,7 +749,7 @@ struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size)</div> <div class="ContentPasted0">> * as WC on CPU side and UC (PAT index 2) on GPU side</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> if (IS_METEORLAKE(gt->i915))</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);</div> <div class="ContentPasted0">> if (IS_ERR(vma))</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c</div> <div class="ContentPasted0">> index d408856ae4c0..e099414d624d 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c</div> <div class="ContentPasted0">> @@ -991,14 +991,10 @@ static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> if (ggtt->vm.raw_insert_entries)</div> <div class="ContentPasted0">> ggtt->vm.raw_insert_entries(&ggtt->vm, vma_res,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(ggtt->vm.i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - pte_flags);</div> <div class="ContentPasted0">> + ggtt->vm.i915->pat_uc, pte_flags);</div> <div class="ContentPasted0">> else</div> <div class="ContentPasted0">> ggtt->vm.insert_entries(&ggtt->vm, vma_res,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(ggtt->vm.i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - pte_flags);</div> <div class="ContentPasted0">> + ggtt->vm.i915->pat_uc, pte_flags);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static void uc_fw_unbind_ggtt(struct intel_uc_fw *uc_fw)</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_cache.c b/drivers/gpu/drm/i915/i915_cache.c</div> <div class="ContentPasted0">> new file mode 100644</div> <div class="ContentPasted0">> index 000000000000..4d0033bf1658</div> <div class="ContentPasted0">> --- /dev/null</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_cache.c</div> <div class="ContentPasted0">> @@ -0,0 +1,72 @@</div> <div class="ContentPasted0">> +/*</div> <div class="ContentPasted0">> + * SPDX-License-Identifier: MIT</div> <div class="ContentPasted0">> + *</div> <div class="ContentPasted0">> + * Copyright � 2023 Intel Corporation</div> <div class="ContentPasted0">> + */</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#include <drm/drm_print.h></div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> +#include "i915_drv.h"</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +int i915_cache_init(struct drm_i915_private *i915)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + int ret;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + ret = i915_cache_find_pat(i915, I915_CACHE(UC));</div> <div class="ContentPasted0">> + if (ret < 0)</div> <div class="ContentPasted0">> + return -ENODEV;</div> <div class="ContentPasted0">> + drm_info(&i915->drm, "Using PAT index %u for uncached access\n", ret);</div> <div class="ContentPasted0">> + i915->pat_uc = ret;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + ret = i915_cache_find_pat(i915, I915_CACHE(WB));</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">For MTL, this would find pat index 0, but KMD wants pat index 3.</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> + if (ret < 0)</div> <div class="ContentPasted0">> + return -ENODEV;</div> <div class="ContentPasted0">> + drm_info(&i915->drm, "Using PAT index %u for write-back access\n", ret);</div> <div class="ContentPasted0">> + i915->pat_wb = ret;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + return 0;</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +int i915_cache_find_pat(struct drm_i915_private *i915, i915_cache_t cache)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + const struct intel_device_info *info = INTEL_INFO(i915);</div> <div class="ContentPasted0">> + int i;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + for (i = 0; i < ARRAY_SIZE(info->cache_modes); i++) {</div> <div class="ContentPasted0">> + if (info->cache_modes[i] == cache)</div> <div class="ContentPasted0">> + return i;</div> <div class="ContentPasted0">> + }</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + return -1;</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +void i915_cache_print(struct drm_printer *p, i915_cache_t cache)</div> <div class="ContentPasted0">> +{</div> <div class="ContentPasted0">> + const enum i915_cache_mode mode = I915_CACHE_MODE(cache);</div> <div class="ContentPasted0">> + const unsigned long flags = I915_CACHE_FLAGS(cache);</div> <div class="ContentPasted0">> + static const char *mode_str[] = {</div> <div class="ContentPasted0">> + [I915_CACHE_MODE_UC] = "UC",</div> <div class="ContentPasted0">> + [I915_CACHE_MODE_WB] = "WB",</div> <div class="ContentPasted0">> + [I915_CACHE_MODE_WT] = "WT",</div> <div class="ContentPasted0">> + [I915_CACHE_MODE_WC] = "WC",</div> <div class="ContentPasted0">> + };</div> <div class="ContentPasted0">> + static const char *flag_str[] = {</div> <div class="ContentPasted0">> + [I915_CACHE_FLAG_L3] = "L3",</div> <div class="ContentPasted0">> + [I915_CACHE_FLAG_COH1W] = "1-Way-Coherent",</div> <div class="ContentPasted0">> + [I915_CACHE_FLAG_COH2W] = "2-Way-Coherent",</div> <div class="ContentPasted0">> + [I915_CACHE_FLAG_CLOS1] = "CLOS1",</div> <div class="ContentPasted0">> + [I915_CACHE_FLAG_CLOS2] = "CLOS2",</div> <div class="ContentPasted0">> + };</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + if (mode == I915_CACHE_MODE_UNKNOWN || mode > ARRAY_SIZE(mode_str)) {</div> <div class="ContentPasted0">> + drm_printf(p, "0x%x", cache);</div> <div class="ContentPasted0">> + } else {</div> <div class="ContentPasted0">> + unsigned long bit;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + drm_printf(p, "%s", mode_str[mode]);</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + for_each_set_bit(bit, &flags, sizeof(i915_cache_t))</div> <div class="ContentPasted0">> + drm_printf(p, "-%s", flag_str[bit]);</div> <div class="ContentPasted0">> + }</div> <div class="ContentPasted0">> +}</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_cache.h b/drivers/gpu/drm/i915/i915_cache.h</div> <div class="ContentPasted0">> new file mode 100644</div> <div class="ContentPasted0">> index 000000000000..ffb621d8a8a0</div> <div class="ContentPasted0">> --- /dev/null</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_cache.h</div> <div class="ContentPasted0">> @@ -0,0 +1,49 @@</div> <div class="ContentPasted0">> +/* SPDX-License-Identifier: MIT */</div> <div class="ContentPasted0">> +/*</div> <div class="ContentPasted0">> + * Copyright � 2023 Intel Corporation</div> <div class="ContentPasted0">> + */</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#ifndef __I915_CACHE_H__</div> <div class="ContentPasted0">> +#define __I915_CACHE_H__</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#include <linux/types.h></div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +struct drm_printer;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +struct drm_i915_private;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +typedef u16 i915_cache_t;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#define I915_CACHE(mode) \</div> <div class="ContentPasted0">> + (i915_cache_t)(I915_CACHE_MODE_##mode)</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#define _I915_CACHE(mode, flag) \</div> <div class="ContentPasted0">> + (i915_cache_t)((I915_CACHE_MODE_##mode) | \</div> <div class="ContentPasted0">> + (BIT(8 + I915_CACHE_FLAG_##flag)))</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#define I915_CACHE_MODE(cache) \</div> <div class="ContentPasted0">> + (enum i915_cache_mode)(((i915_cache_t)(cache)) & 0xff)</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAGS(cache) \</div> <div class="ContentPasted0">> + (unsigned int)((((i915_cache_t)(cache) & 0xff00)) >> 16)</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +/* Cache modes */</div> <div class="ContentPasted0">> +enum i915_cache_mode {</div> <div class="ContentPasted0">> + I915_CACHE_MODE_UNKNOWN = 0,</div> <div class="ContentPasted0">> + I915_CACHE_MODE_UC,</div> <div class="ContentPasted0">> + I915_CACHE_MODE_WB,</div> <div class="ContentPasted0">> + I915_CACHE_MODE_WT,</div> <div class="ContentPasted0">> + I915_CACHE_MODE_WC</div> <div class="ContentPasted0">> +};</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +/* Cache mode flag bits */</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAG_L3 (0)</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAG_COH1W (1)</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAG_COH2W (2)</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAG_CLOS1 (3)</div> <div class="ContentPasted0">> +#define I915_CACHE_FLAG_CLOS2 (4)</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +int i915_cache_init(struct drm_i915_private *i915);</div> <div class="ContentPasted0">> +int i915_cache_find_pat(struct drm_i915_private *i915, i915_cache_t cache);</div> <div class="ContentPasted0">> +void i915_cache_print(struct drm_printer *p, i915_cache_t cache);</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> +#endif /* __I915_CACHE_H__ */</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c</div> <div class="ContentPasted0">> index 4de44cf1026d..03d366e0bdf7 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_debugfs.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_debugfs.c</div> <div class="ContentPasted0">> @@ -140,68 +140,33 @@ static const char *stringify_vma_type(const struct i915_vma *vma)</div> <div class="ContentPasted0">> return "ppgtt";</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -static const char *i915_cache_level_str(struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> -{</div> <div class="ContentPasted0">> - struct drm_i915_private *i915 = obj_to_i915(obj);</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> - if (IS_METEORLAKE(i915)) {</div> <div class="ContentPasted0">> - switch (obj->pat_index) {</div> <div class="ContentPasted0">> - case 0: return " WB";</div> <div class="ContentPasted0">> - case 1: return " WT";</div> <div class="ContentPasted0">> - case 2: return " UC";</div> <div class="ContentPasted0">> - case 3: return " WB (1-Way Coh)";</div> <div class="ContentPasted0">> - case 4: return " WB (2-Way Coh)";</div> <div class="ContentPasted0">> - default: return " not defined";</div> <div class="ContentPasted0">> - }</div> <div class="ContentPasted0">> - } else if (IS_PONTEVECCHIO(i915)) {</div> <div class="ContentPasted0">> - switch (obj->pat_index) {</div> <div class="ContentPasted0">> - case 0: return " UC";</div> <div class="ContentPasted0">> - case 1: return " WC";</div> <div class="ContentPasted0">> - case 2: return " WT";</div> <div class="ContentPasted0">> - case 3: return " WB";</div> <div class="ContentPasted0">> - case 4: return " WT (CLOS1)";</div> <div class="ContentPasted0">> - case 5: return " WB (CLOS1)";</div> <div class="ContentPasted0">> - case 6: return " WT (CLOS2)";</div> <div class="ContentPasted0">> - case 7: return " WT (CLOS2)";</div> <div class="ContentPasted0">> - default: return " not defined";</div> <div class="ContentPasted0">> - }</div> <div class="ContentPasted0">> - } else if (GRAPHICS_VER(i915) >= 12) {</div> <div class="ContentPasted0">> - switch (obj->pat_index) {</div> <div class="ContentPasted0">> - case 0: return " WB";</div> <div class="ContentPasted0">> - case 1: return " WC";</div> <div class="ContentPasted0">> - case 2: return " WT";</div> <div class="ContentPasted0">> - case 3: return " UC";</div> <div class="ContentPasted0">> - default: return " not defined";</div> <div class="ContentPasted0">> - }</div> <div class="ContentPasted0">> - } else {</div> <div class="ContentPasted0">> - switch (obj->pat_index) {</div> <div class="ContentPasted0">> - case 0: return " UC";</div> <div class="ContentPasted0">> - case 1: return HAS_LLC(i915) ?</div> <div class="ContentPasted0">> - " LLC" : " snooped";</div> <div class="ContentPasted0">> - case 2: return " L3+LLC";</div> <div class="ContentPasted0">> - case 3: return " WT";</div> <div class="ContentPasted0">> - default: return " not defined";</div> <div class="ContentPasted0">> - }</div> <div class="ContentPasted0">> - }</div> <div class="ContentPasted0">> -}</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> void</div> <div class="ContentPasted0">> i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)</div> <div class="ContentPasted0">> {</div> <div class="ContentPasted0">> + struct drm_i915_private *i915 = to_i915(obj->base.dev);</div> <div class="ContentPasted0">> + struct drm_printer p = drm_seq_file_printer(m);</div> <div class="ContentPasted0">> struct i915_vma *vma;</div> <div class="ContentPasted0">> int pin_count = 0;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - seq_printf(m, "%pK: %c%c%c %8zdKiB %02x %02x %s%s%s",</div> <div class="ContentPasted0">> + seq_printf(m, "%pK: %c%c%c %8zdKiB %02x %02x ",</div> <div class="ContentPasted0">> &obj->base,</div> <div class="ContentPasted0">> get_tiling_flag(obj),</div> <div class="ContentPasted0">> get_global_flag(obj),</div> <div class="ContentPasted0">> get_pin_mapped_flag(obj),</div> <div class="ContentPasted0">> obj->base.size / 1024,</div> <div class="ContentPasted0">> obj->read_domains,</div> <div class="ContentPasted0">> - obj->write_domain,</div> <div class="ContentPasted0">> - i915_cache_level_str(obj),</div> <div class="ContentPasted0">> - obj->mm.dirty ? " dirty" : "",</div> <div class="ContentPasted0">> - obj->mm.madv == I915_MADV_DONTNEED ? " purgeable" : "");</div> <div class="ContentPasted0">> + obj->write_domain);</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + i915_cache_print(&p, INTEL_INFO(i915)->cache_modes[obj->pat_index]);</div> <div class="ContentPasted0">> + if (obj->pat_set_by_user)</div> <div class="ContentPasted0">> + drm_puts(&p, "!");</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + if (obj->mm.dirty)</div> <div class="ContentPasted0">> + seq_puts(m, " dirty");</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> + if (obj->mm.madv == I915_MADV_DONTNEED)</div> <div class="ContentPasted0">> + seq_puts(m, " purgeable");</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> if (obj->base.name)</div> <div class="ContentPasted0">> seq_printf(m, " (name: %d)", obj->base.name);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c</div> <div class="ContentPasted0">> index 294b022de22b..8663388a524f 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_driver.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_driver.c</div> <div class="ContentPasted0">> @@ -80,6 +80,7 @@</div> <div class="ContentPasted0">> #include "soc/intel_dram.h"</div> <div class="ContentPasted0">> #include "soc/intel_gmch.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> #include "i915_debugfs.h"</div> <div class="ContentPasted0">> #include "i915_driver.h"</div> <div class="ContentPasted0">> #include "i915_drm_client.h"</div> <div class="ContentPasted0">> @@ -240,6 +241,10 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)</div> <div class="ContentPasted0">> i915_memcpy_init_early(dev_priv);</div> <div class="ContentPasted0">> intel_runtime_pm_init_early(&dev_priv->runtime_pm);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> + ret = i915_cache_init(dev_priv);</div> <div class="ContentPasted0">> + if (ret < 0)</div> <div class="ContentPasted0">> + return ret;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> ret = i915_workqueues_init(dev_priv);</div> <div class="ContentPasted0">> if (ret < 0)</div> <div class="ContentPasted0">> return ret;</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h</div> <div class="ContentPasted0">> index a18013787c9f..462b076e2650 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_drv.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_drv.h</div> <div class="ContentPasted0">> @@ -250,6 +250,9 @@ struct drm_i915_private {</div> <div class="ContentPasted0">> unsigned int hpll_freq;</div> <div class="ContentPasted0">> unsigned int czclk_freq;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> + unsigned int pat_uc;</div> <div class="ContentPasted0">> + unsigned int pat_wb;</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> /**</div> <div class="ContentPasted0">> * wq - Driver workqueue for GEM.</div> <div class="ContentPasted0">> *</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c</div> <div class="ContentPasted0">> index b46d7a2dab1a..ef6d33c2c288 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_gem.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_gem.c</div> <div class="ContentPasted0">> @@ -422,9 +422,7 @@ i915_gem_gtt_pread(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> ggtt->vm.insert_page(&ggtt->vm,</div> <div class="ContentPasted0">> i915_gem_object_get_dma_address(obj,</div> <div class="ContentPasted0">> offset >> PAGE_SHIFT),</div> <div class="ContentPasted0">> - node.start,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE), 0);</div> <div class="ContentPasted0">> + node.start, i915->pat_uc, 0);</div> <div class="ContentPasted0">> } else {</div> <div class="ContentPasted0">> page_base += offset & PAGE_MASK;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> @@ -603,9 +601,7 @@ i915_gem_gtt_pwrite_fast(struct drm_i915_gem_object *obj,</div> <div class="ContentPasted0">> ggtt->vm.insert_page(&ggtt->vm,</div> <div class="ContentPasted0">> i915_gem_object_get_dma_address(obj,</div> <div class="ContentPasted0">> offset >> PAGE_SHIFT),</div> <div class="ContentPasted0">> - node.start,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE), 0);</div> <div class="ContentPasted0">> + node.start, i915->pat_uc, 0);</div> <div class="ContentPasted0">> wmb(); /* flush modifications to the GGTT (insert_page) */</div> <div class="ContentPasted0">> } else {</div> <div class="ContentPasted0">> page_base += offset & PAGE_MASK;</div> <div class="ContentPasted0">> @@ -1148,19 +1144,6 @@ int i915_gem_init(struct drm_i915_private *dev_priv)</div> <div class="ContentPasted0">> unsigned int i;</div> <div class="ContentPasted0">> int ret;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /*</div> <div class="ContentPasted0">> - * In the proccess of replacing cache_level with pat_index a tricky</div> <div class="ContentPasted0">> - * dependency is created on the definition of the enum i915_cache_level.</div> <div class="ContentPasted0">> - * in case this enum is changed, PTE encode would be broken.</div> <div class="ContentPasted0">> - * Add a WARNING here. And remove when we completely quit using this</div> <div class="ContentPasted0">> - * enum</div> <div class="ContentPasted0">> - */</div> <div class="ContentPasted0">> - BUILD_BUG_ON(I915_CACHE_NONE != 0 ||</div> <div class="ContentPasted0">> - I915_CACHE_LLC != 1 ||</div> <div class="ContentPasted0">> - I915_CACHE_L3_LLC != 2 ||</div> <div class="ContentPasted0">> - I915_CACHE_WT != 3 ||</div> <div class="ContentPasted0">> - I915_MAX_CACHE_LEVEL != 4);</div> <div class="ContentPasted0">> -</div> <div class="ContentPasted0">> /* We need to fallback to 4K pages if host doesn't support huge gtt. */</div> <div class="ContentPasted0">> if (intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv))</div> <div class="ContentPasted0">> RUNTIME_INFO(dev_priv)->page_sizes = I915_GTT_PAGE_SIZE_4K;</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c</div> <div class="ContentPasted0">> index 92ca81f9feab..6a6f97aaec60 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_gpu_error.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c</div> <div class="ContentPasted0">> @@ -1124,14 +1124,11 @@ i915_vma_coredump_create(const struct intel_gt *gt,</div> <div class="ContentPasted0">> mutex_lock(&ggtt->error_mutex);</div> <div class="ContentPasted0">> if (ggtt->vm.raw_insert_page)</div> <div class="ContentPasted0">> ggtt->vm.raw_insert_page(&ggtt->vm, dma, slot,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + ggtt->vm.i915->pat_uc,</div> <div class="ContentPasted0">> 0);</div> <div class="ContentPasted0">> else</div> <div class="ContentPasted0">> ggtt->vm.insert_page(&ggtt->vm, dma, slot,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(gt->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + ggtt->vm.i915->pat_uc, 0);</div> <div class="ContentPasted0">> mb();</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> s = io_mapping_map_wc(&ggtt->iomap, slot, PAGE_SIZE);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c</div> <div class="ContentPasted0">> index dea3609fc496..c0c6248fbc78 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_pci.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_pci.c</div> <div class="ContentPasted0">> @@ -32,6 +32,7 @@</div> <div class="ContentPasted0">> #include "gt/intel_sa_media.h"</div> <div class="ContentPasted0">> #include "gem/i915_gem_object_types.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> #include "i915_driver.h"</div> <div class="ContentPasted0">> #include "i915_drv.h"</div> <div class="ContentPasted0">> #include "i915_pci.h"</div> <div class="ContentPasted0">> @@ -46,36 +47,42 @@ __diag_ignore_all("-Woverride-init", "Allow overriding inherited members");</div> <div class="ContentPasted0">> .__runtime.graphics.ip.ver = (x), \</div> <div class="ContentPasted0">> .__runtime.media.ip.ver = (x)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -#define LEGACY_CACHELEVEL \</div> <div class="ContentPasted0">> - .cachelevel_to_pat = { \</div> <div class="ContentPasted0">> - [I915_CACHE_NONE] = 0, \</div> <div class="ContentPasted0">> - [I915_CACHE_LLC] = 1, \</div> <div class="ContentPasted0">> - [I915_CACHE_L3_LLC] = 2, \</div> <div class="ContentPasted0">> - [I915_CACHE_WT] = 3, \</div> <div class="ContentPasted0">> +/* TODO/QQQ index 1 & 2 */</div> <div class="ContentPasted0">> +#define LEGACY_CACHE_MODES \</div> <div class="ContentPasted0">> + .cache_modes = { \</div> <div class="ContentPasted0">> + [0] = I915_CACHE(UC), \</div> <div class="ContentPasted0">> + [1] = I915_CACHE(WB), \</div> <div class="ContentPasted0">> + [2] = _I915_CACHE(WB, L3), \</div> <div class="ContentPasted0">> + [3] = I915_CACHE(WT), \</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -#define TGL_CACHELEVEL \</div> <div class="ContentPasted0">> - .cachelevel_to_pat = { \</div> <div class="ContentPasted0">> - [I915_CACHE_NONE] = 3, \</div> <div class="ContentPasted0">> - [I915_CACHE_LLC] = 0, \</div> <div class="ContentPasted0">> - [I915_CACHE_L3_LLC] = 0, \</div> <div class="ContentPasted0">> - [I915_CACHE_WT] = 2, \</div> <div class="ContentPasted0">> +#define GEN12_CACHE_MODES \</div> <div class="ContentPasted0">> + .cache_modes = { \</div> <div class="ContentPasted0">> + [0] = I915_CACHE(WB), \</div> <div class="ContentPasted0">> + [1] = I915_CACHE(WC), \</div> <div class="ContentPasted0">> + [2] = I915_CACHE(WT), \</div> <div class="ContentPasted0">> + [3] = I915_CACHE(UC), \</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -#define PVC_CACHELEVEL \</div> <div class="ContentPasted0">> - .cachelevel_to_pat = { \</div> <div class="ContentPasted0">> - [I915_CACHE_NONE] = 0, \</div> <div class="ContentPasted0">> - [I915_CACHE_LLC] = 3, \</div> <div class="ContentPasted0">> - [I915_CACHE_L3_LLC] = 3, \</div> <div class="ContentPasted0">> - [I915_CACHE_WT] = 2, \</div> <div class="ContentPasted0">> +#define PVC_CACHE_MODES \</div> <div class="ContentPasted0">> + .cache_modes = { \</div> <div class="ContentPasted0">> + [0] = I915_CACHE(UC), \</div> <div class="ContentPasted0">> + [1] = I915_CACHE(WC), \</div> <div class="ContentPasted0">> + [2] = I915_CACHE(WT), \</div> <div class="ContentPasted0">> + [3] = I915_CACHE(WB), \</div> <div class="ContentPasted0">> + [4] = _I915_CACHE(WT, CLOS1), \</div> <div class="ContentPasted0">> + [5] = _I915_CACHE(WB, CLOS1), \</div> <div class="ContentPasted0">> + [6] = _I915_CACHE(WT, CLOS2), \</div> <div class="ContentPasted0">> + [7] = _I915_CACHE(WB, CLOS2), \</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> -#define MTL_CACHELEVEL \</div> <div class="ContentPasted0">> - .cachelevel_to_pat = { \</div> <div class="ContentPasted0">> - [I915_CACHE_NONE] = 2, \</div> <div class="ContentPasted0">> - [I915_CACHE_LLC] = 3, \</div> <div class="ContentPasted0">> - [I915_CACHE_L3_LLC] = 3, \</div> <div class="ContentPasted0">> - [I915_CACHE_WT] = 1, \</div> <div class="ContentPasted0">> +#define MTL_CACHE_MODES \</div> <div class="ContentPasted0">> + .cache_modes = { \</div> <div class="ContentPasted0">> + [0] = I915_CACHE(WB), \</div> <div class="ContentPasted0">> + [1] = I915_CACHE(WT), \</div> <div class="ContentPasted0">> + [2] = I915_CACHE(UC), \</div> <div class="ContentPasted0">> + [3] = _I915_CACHE(WB, COH1W), \</div> <div class="ContentPasted0">> + [4] = _I915_CACHE(WB, COH2W), \</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* Keep in gen based order, and chronological order within a gen */</div> <div class="ContentPasted0">> @@ -100,7 +107,7 @@ __diag_ignore_all("-Woverride-init", "Allow overriding inherited members");</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define I845_FEATURES \</div> <div class="ContentPasted0">> GEN(2), \</div> <div class="ContentPasted0">> @@ -115,7 +122,7 @@ __diag_ignore_all("-Woverride-init", "Allow overriding inherited members");</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_device_info i830_info = {</div> <div class="ContentPasted0">> I830_FEATURES,</div> <div class="ContentPasted0">> @@ -148,7 +155,7 @@ static const struct intel_device_info i865g_info = {</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_device_info i915g_info = {</div> <div class="ContentPasted0">> GEN3_FEATURES,</div> <div class="ContentPasted0">> @@ -211,7 +218,7 @@ static const struct intel_device_info pnv_m_info = {</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_device_info i965g_info = {</div> <div class="ContentPasted0">> GEN4_FEATURES,</div> <div class="ContentPasted0">> @@ -255,7 +262,7 @@ static const struct intel_device_info gm45_info = {</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_device_info ilk_d_info = {</div> <div class="ContentPasted0">> GEN5_FEATURES,</div> <div class="ContentPasted0">> @@ -285,7 +292,7 @@ static const struct intel_device_info ilk_m_info = {</div> <div class="ContentPasted0">> .__runtime.ppgtt_size = 31, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define SNB_D_PLATFORM \</div> <div class="ContentPasted0">> GEN6_FEATURES, \</div> <div class="ContentPasted0">> @@ -333,7 +340,7 @@ static const struct intel_device_info snb_m_gt2_info = {</div> <div class="ContentPasted0">> .__runtime.ppgtt_size = 31, \</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define IVB_D_PLATFORM \</div> <div class="ContentPasted0">> GEN7_FEATURES, \</div> <div class="ContentPasted0">> @@ -390,7 +397,7 @@ static const struct intel_device_info vlv_info = {</div> <div class="ContentPasted0">> .platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0),</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES,</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS,</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL,</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define G75_FEATURES \</div> <div class="ContentPasted0">> @@ -476,7 +483,7 @@ static const struct intel_device_info chv_info = {</div> <div class="ContentPasted0">> .has_coherent_ggtt = false,</div> <div class="ContentPasted0">> GEN_DEFAULT_PAGE_SIZES,</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS,</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL,</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #define GEN9_DEFAULT_PAGE_SIZES \</div> <div class="ContentPasted0">> @@ -539,7 +546,7 @@ static const struct intel_device_info skl_gt4_info = {</div> <div class="ContentPasted0">> .max_pat_index = 3, \</div> <div class="ContentPasted0">> GEN9_DEFAULT_PAGE_SIZES, \</div> <div class="ContentPasted0">> GEN_DEFAULT_REGIONS, \</div> <div class="ContentPasted0">> - LEGACY_CACHELEVEL</div> <div class="ContentPasted0">> + LEGACY_CACHE_MODES</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_device_info bxt_info = {</div> <div class="ContentPasted0">> GEN9_LP_FEATURES,</div> <div class="ContentPasted0">> @@ -643,7 +650,7 @@ static const struct intel_device_info jsl_info = {</div> <div class="ContentPasted0">> #define GEN12_FEATURES \</div> <div class="ContentPasted0">> GEN11_FEATURES, \</div> <div class="ContentPasted0">> GEN(12), \</div> <div class="ContentPasted0">> - TGL_CACHELEVEL, \</div> <div class="ContentPasted0">> + GEN12_CACHE_MODES, \</div> <div class="ContentPasted0">> .has_global_mocs = 1, \</div> <div class="ContentPasted0">> .has_pxp = 1, \</div> <div class="ContentPasted0">> .max_pat_index = 3</div> <div class="ContentPasted0">> @@ -711,7 +718,7 @@ static const struct intel_device_info adl_p_info = {</div> <div class="ContentPasted0">> .__runtime.graphics.ip.ver = 12, \</div> <div class="ContentPasted0">> .__runtime.graphics.ip.rel = 50, \</div> <div class="ContentPasted0">> XE_HP_PAGE_SIZES, \</div> <div class="ContentPasted0">> - TGL_CACHELEVEL, \</div> <div class="ContentPasted0">> + GEN12_CACHE_MODES, \</div> <div class="ContentPasted0">> .dma_mask_size = 46, \</div> <div class="ContentPasted0">> .has_3d_pipeline = 1, \</div> <div class="ContentPasted0">> .has_64bit_reloc = 1, \</div> <div class="ContentPasted0">> @@ -806,7 +813,7 @@ static const struct intel_device_info pvc_info = {</div> <div class="ContentPasted0">> BIT(VCS0) |</div> <div class="ContentPasted0">> BIT(CCS0) | BIT(CCS1) | BIT(CCS2) | BIT(CCS3),</div> <div class="ContentPasted0">> .require_force_probe = 1,</div> <div class="ContentPasted0">> - PVC_CACHELEVEL,</div> <div class="ContentPasted0">> + PVC_CACHE_MODES</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> static const struct intel_gt_definition xelpmp_extra_gt[] = {</div> <div class="ContentPasted0">> @@ -841,7 +848,7 @@ static const struct intel_device_info mtl_info = {</div> <div class="ContentPasted0">> .memory_regions = REGION_SMEM | REGION_STOLEN_LMEM,</div> <div class="ContentPasted0">> .platform_engine_mask = BIT(RCS0) | BIT(BCS0) | BIT(CCS0),</div> <div class="ContentPasted0">> .require_force_probe = 1,</div> <div class="ContentPasted0">> - MTL_CACHELEVEL,</div> <div class="ContentPasted0">> + MTL_CACHE_MODES</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #undef PLATFORM</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c</div> <div class="ContentPasted0">> index 0a111b281578..1b8e067f37dd 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/i915_perf.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/i915_perf.c</div> <div class="ContentPasted0">> @@ -1861,7 +1861,7 @@ static int alloc_oa_buffer(struct i915_perf_stream *stream)</div> <div class="ContentPasted0">> return PTR_ERR(bo);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(bo, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(bo, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* PreHSW required 512K alignment, HSW requires 16M */</div> <div class="ContentPasted0">> vma = i915_vma_instance(bo, &gt->ggtt->vm, NULL);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h</div> <div class="ContentPasted0">> index dbfe6443457b..f48a21895a85 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/intel_device_info.h</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/intel_device_info.h</div> <div class="ContentPasted0">> @@ -27,6 +27,8 @@</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #include <uapi/drm/i915_drm.h></div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> +#include "i915_cache.h"</div> <div class="ContentPasted0">> +</div> <div class="ContentPasted0">> #include "intel_step.h"</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> #include "gt/intel_engine_types.h"</div> <div class="ContentPasted0">> @@ -243,8 +245,8 @@ struct intel_device_info {</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> const struct intel_runtime_info __runtime;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - u32 cachelevel_to_pat[I915_MAX_CACHE_LEVEL];</div> <div class="ContentPasted0">> - u32 max_pat_index;</div> <div class="ContentPasted0">> + i915_cache_t cache_modes[9];</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">I was commenting on the array size here. It's probably better to make it 16</div> <div class="ContentPasted0">because there are 4 PAT index bits defined in the PTE. Indices above max_pat_index</div> <div class="ContentPasted0">are not used, but theoretically new mode could be added. Well, it's up to you,</div> <div class="ContentPasted0">not likely to happen anyway.</div> <div><br class="ContentPasted0"> </div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">-Fei</div> <div><br class="ContentPasted0"> </div> <div class="ContentPasted0">> + unsigned int max_pat_index;</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> struct intel_driver_caps {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c</div> <div class="ContentPasted0">> index 61da4ed9d521..e620f73793a5 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/i915_gem.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/i915_gem.c</div> <div class="ContentPasted0">> @@ -57,10 +57,7 @@ static void trash_stolen(struct drm_i915_private *i915)</div> <div class="ContentPasted0">> u32 __iomem *s;</div> <div class="ContentPasted0">> int x;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - ggtt->vm.insert_page(&ggtt->vm, dma, slot,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + ggtt->vm.insert_page(&ggtt->vm, dma, slot, i915->pat_uc, 0);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> s = io_mapping_map_atomic_wc(&ggtt->iomap, slot);</div> <div class="ContentPasted0">> for (x = 0; x < PAGE_SIZE / sizeof(u32); x++) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c</div> <div class="ContentPasted0">> index f8fe3681c3dc..862d3ba0e025 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c</div> <div class="ContentPasted0">> @@ -246,7 +246,7 @@ static int igt_evict_for_cache_color(void *arg)</div> <div class="ContentPasted0">> struct drm_mm_node target = {</div> <div class="ContentPasted0">> .start = I915_GTT_PAGE_SIZE * 2,</div> <div class="ContentPasted0">> .size = I915_GTT_PAGE_SIZE,</div> <div class="ContentPasted0">> - .color = i915_gem_get_pat_index(gt->i915, I915_CACHE_LLC),</div> <div class="ContentPasted0">> + .color = I915_CACHE(WB),</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> struct drm_i915_gem_object *obj;</div> <div class="ContentPasted0">> struct i915_vma *vma;</div> <div class="ContentPasted0">> @@ -267,7 +267,7 @@ static int igt_evict_for_cache_color(void *arg)</div> <div class="ContentPasted0">> err = PTR_ERR(obj);</div> <div class="ContentPasted0">> goto cleanup;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> quirk_add(obj, &objects);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,</div> <div class="ContentPasted0">> @@ -283,7 +283,7 @@ static int igt_evict_for_cache_color(void *arg)</div> <div class="ContentPasted0">> err = PTR_ERR(obj);</div> <div class="ContentPasted0">> goto cleanup;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(WB));</div> <div class="ContentPasted0">> quirk_add(obj, &objects);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* Neighbouring; same colour - should fit */</div> <div class="ContentPasted0">> @@ -309,7 +309,7 @@ static int igt_evict_for_cache_color(void *arg)</div> <div class="ContentPasted0">> /* Attempt to remove the first *pinned* vma, by removing the (empty)</div> <div class="ContentPasted0">> * neighbour -- this should fail.</div> <div class="ContentPasted0">> */</div> <div class="ContentPasted0">> - target.color = i915_gem_get_pat_index(gt->i915, I915_CACHE_L3_LLC);</div> <div class="ContentPasted0">> + target.color = _I915_CACHE(WB, L3);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> mutex_lock(&ggtt->vm.mutex);</div> <div class="ContentPasted0">> err = i915_gem_evict_for_node(&ggtt->vm, NULL, &target, 0);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c</div> <div class="ContentPasted0">> index 5c397a2df70e..a24585784f75 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c</div> <div class="ContentPasted0">> @@ -135,7 +135,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->write_domain = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU;</div> <div class="ContentPasted0">> - obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + obj->pat_index = i915->pat_uc;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> /* Preallocate the "backing storage" */</div> <div class="ContentPasted0">> if (i915_gem_object_pin_pages_unlocked(obj))</div> <div class="ContentPasted0">> @@ -358,10 +358,8 @@ static int lowlevel_hole(struct i915_address_space *vm,</div> <div class="ContentPasted0">> mock_vma_res->start = addr;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> with_intel_runtime_pm(vm->gt->uncore->rpm, wakeref)</div> <div class="ContentPasted0">> - vm->insert_entries(vm, mock_vma_res,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(vm->i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + vm->insert_entries(vm, mock_vma_res,</div> <div class="ContentPasted0">> + vm->i915->pat_uc, 0);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> count = n;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> @@ -1379,10 +1377,7 @@ static int igt_ggtt_page(void *arg)</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> ggtt->vm.insert_page(&ggtt->vm,</div> <div class="ContentPasted0">> i915_gem_object_get_dma_address(obj, 0),</div> <div class="ContentPasted0">> - offset,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> - 0);</div> <div class="ContentPasted0">> + offset, ggtt->vm.i915->pat_uc, 0);</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> order = i915_random_order(count, &prng);</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/igt_spinner.c b/drivers/gpu/drm/i915/selftests/igt_spinner.c</div> <div class="ContentPasted0">> index 618d9386d554..d42d1bcffe21 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/igt_spinner.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/igt_spinner.c</div> <div class="ContentPasted0">> @@ -23,7 +23,7 @@ int igt_spinner_init(struct igt_spinner *spin, struct intel_gt *gt)</div> <div class="ContentPasted0">> err = PTR_ERR(spin->hws);</div> <div class="ContentPasted0">> goto err;</div> <div class="ContentPasted0">> }</div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(spin->hws, I915_CACHE_LLC);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(spin->hws, I915_CACHE(WB));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> spin->obj = i915_gem_object_create_internal(gt->i915, PAGE_SIZE);</div> <div class="ContentPasted0">> if (IS_ERR(spin->obj)) {</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c</div> <div class="ContentPasted0">> index d985d9bae2e8..b82fe0ef8cd7 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c</div> <div class="ContentPasted0">> @@ -1070,9 +1070,7 @@ static int igt_lmem_write_cpu(void *arg)</div> <div class="ContentPasted0">> /* Put the pages into a known state -- from the gpu for added fun */</div> <div class="ContentPasted0">> intel_engine_pm_get(engine);</div> <div class="ContentPasted0">> err = intel_context_migrate_clear(engine->gt->migrate.context, NULL,</div> <div class="ContentPasted0">> - obj->mm.pages->sgl,</div> <div class="ContentPasted0">> - i915_gem_get_pat_index(i915,</div> <div class="ContentPasted0">> - I915_CACHE_NONE),</div> <div class="ContentPasted0">> + obj->mm.pages->sgl, i915->pat_uc,</div> <div class="ContentPasted0">> true, 0xdeadbeaf, &rq);</div> <div class="ContentPasted0">> if (rq) {</div> <div class="ContentPasted0">> dma_resv_add_fence(obj->base.resv, &rq->fence,</div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/mock_gem_device.c b/drivers/gpu/drm/i915/selftests/mock_gem_device.c</div> <div class="ContentPasted0">> index 12aa7fbb0748..f478a4102299 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/mock_gem_device.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/mock_gem_device.c</div> <div class="ContentPasted0">> @@ -126,13 +126,13 @@ static const struct intel_device_info mock_info = {</div> <div class="ContentPasted0">> .memory_regions = REGION_SMEM,</div> <div class="ContentPasted0">> .platform_engine_mask = BIT(0),</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - /* simply use legacy cache level for mock device */</div> <div class="ContentPasted0">> + /* Simply use legacy cache modes for the mock device. */</div> <div class="ContentPasted0">> .max_pat_index = 3,</div> <div class="ContentPasted0">> - .cachelevel_to_pat = {</div> <div class="ContentPasted0">> - [I915_CACHE_NONE] = 0,</div> <div class="ContentPasted0">> - [I915_CACHE_LLC] = 1,</div> <div class="ContentPasted0">> - [I915_CACHE_L3_LLC] = 2,</div> <div class="ContentPasted0">> - [I915_CACHE_WT] = 3,</div> <div class="ContentPasted0">> + .cache_modes = {</div> <div class="ContentPasted0">> + [0] = I915_CACHE(UC),</div> <div class="ContentPasted0">> + [1] = I915_CACHE(WB),</div> <div class="ContentPasted0">> + [2] = _I915_CACHE(WB, L3),</div> <div class="ContentPasted0">> + [3] = I915_CACHE(WT),</div> <div class="ContentPasted0">> },</div> <div class="ContentPasted0">> };</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> diff --git a/drivers/gpu/drm/i915/selftests/mock_region.c b/drivers/gpu/drm/i915/selftests/mock_region.c</div> <div class="ContentPasted0">> index 6324eb32d4dd..8c16202e79cb 100644</div> <div class="ContentPasted0">> --- a/drivers/gpu/drm/i915/selftests/mock_region.c</div> <div class="ContentPasted0">> +++ b/drivers/gpu/drm/i915/selftests/mock_region.c</div> <div class="ContentPasted0">> @@ -77,7 +77,7 @@ static int mock_object_init(struct intel_memory_region *mem,</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> obj->read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT;</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> - i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE);</div> <div class="ContentPasted0">> + i915_gem_object_set_cache_coherency(obj, I915_CACHE(UC));</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> i915_gem_object_init_memory_region(obj, mem);</div> <div class="ContentPasted0">> </div> <div class="ContentPasted0">> --</div> <div class="ContentPasted0">> 2.39.2</div> <br> </div> </body> </html>