[Intel-gfx] [PATCH 03/20] drm/i915: TDR / per-engine hang recovery support for gen8.
Arun Siluvery
arun.siluvery at linux.intel.com
Wed Jan 13 09:28:15 PST 2016
From: Tomas Elf <tomas.elf at intel.com>
TDR = Timeout Detection and Recovery.
This change introduces support for TDR-style per-engine reset as an initial,
less intrusive hang recovery option to be attempted before falling back to the
legacy full GPU reset recovery mode if necessary. Initially we're only
supporting gen8 but adding support for gen7 is straight-forward since we've
already established an extensible framework where gen7 support can be plugged
in (add corresponding versions of intel_ring_enable, intel_ring_disable,
intel_ring_save, intel_ring_restore, etc.).
1. Per-engine recovery vs. Full GPU recovery
To capture the state of a single engine being detected as hung there is now a
new flag for every engine that can be set once the decision has been made to
schedule hang recovery for that particular engine. This patch only provides the
hang recovery path but not the hang detection integration so for now there is
no way of detecting individual engines as hung and targetting that individual
engine for per-engine hang recovery.
The following algorithm is used to determine when to use which recovery mode
given that hang detection has somehow detected a hang on an individual engine
and given that per-engine hang recovery has been enabled (which it by default
is not):
1. The error handler checks all engines that have been marked as hung
by the hang checker and checks how long ago it was since it last
attempted to do per-engine hang recovery for each respective, currently
hung engine. If the measured time period is within a certain time
window, i.e. the last per-engine hang recovery was done too recently,
it is determined that the previously attempted per-engine hang recovery
was ineffective and the step is taken to promote the current hang to a
full GPU reset. The default value for this time window is 10 seconds,
meaning any hang happening within 10 seconds of a previous hang on the
same engine will be promoted to full GPU reset. (of course, as long as
the per-engine hang recovery option is disabled this won't matter and
the error handler will always go for legacy full GPU reset)
2. If the error handler determines that no currently hung engine has
recently had hang recovery a per-engine hang recovery is scheduled.
3. If the decision to go with per-engine hang recovery is not taken, or
if per-engine hang recovery is attempted but failed for whatever
reason, TDR falls back to legacy full GPU recovery.
NOTE: Gen7 and earlier will always promote to full GPU reset since there is
currently no per-engine reset support for these gens.
2. Context Submission Status Consistency.
Per-engine hang recovery on gen8 (or execlist submission mode in general)
relies on the basic concept of context submission status consistency. What this
means is that we make sure that the status of the hardware and the driver when
it comes to the submission of the currently running context on any engine is
consistent. For example, when submitting a context to the corresponding ELSP
port of an engine we expect the owning request of that context to be at the
head of the corresponding execution list queue. Likewise, as long as the
context is executing on the GPU we expect the EXECLIST_STATUS register and the
context status buffer (CSB) to reflect this. Thus, if the context submission
status is consistent the ID of the currently executing context should be in
EXECLIST_STATUS and it should be consistent with the context of the head
request element in the execution list queue corresponding to that engine.
The reason why this is important for per-engine hang recovery in execlist mode
is because this recovery mode relies on context resubmission in order to resume
execution following the recovery. If a context has been determined to be hung
and the per-engine hang recovery mode is engaged leading to the resubmission of
that context it's important that the hardware is in fact not busy doing
something else or is being idle since a resubmission during this state could
cause unforseen side-effects such as unexpected preemptions.
There are rare, although consistently reproducable, situations that have shown
up in practice where the driver and hardware are no longer consistent with each
other, e.g. due to lost context completion interrupts after which the hardware
would be idle but the driver would still think that a context would still be
active.
3. There is a new reset path for engine reset alongside the legacy full GPU
reset path. This path does the following:
1) Check for context submission consistency to make sure that the
context that the hardware is currently stuck on is actually what the
driver is working on. If not then clearly we're not in a consistently
hung state and we bail out early.
2) Disable/idle the engine. This is done through reset handshaking on
gen8+ unlike earlier gens where this was done by clearing the ring
valid bits in MI_MODE and ring control registers, which are no longer
supported on gen8+. Reset handshaking translates to setting the reset
request bit in the reset control register.
3) Save the current engine state. What this translates to on gen8 is
simply to read the current value of the head register and nudge it so
that it points to the next valid instruction in the ring buffer. Since
we assume that the execution is currently stuck in a batch buffer the
effect of this is that the batchbuffer start instruction of the hung
batch buffer is skipped so that when execution resumes, following the
hang recovery completion, it resumes immediately following the batch
buffer.
This effectively means that we're forcefully terminating the currently
active, hung batch buffer. Obviously, the outcome of this intervention
is potentially undefined but there are not many good options in this
scenario. It's better than resetting the entire GPU in the vast
majority of cases.
Save the nudged head value to be applied later.
4) Reset the engine.
5) Apply the nudged head value to the head register.
6) Reenable the engine. For gen8 this means resubmitting the fixed-up
context, allowing execution to resume. In order to resubmit a context
without relying on the currently hung execlist queue we use a new,
privileged API that is dedicated to TDR use only. This submission API
bypasses any currently queued work and gets exclusive access to the
ELSP ports.
7) If the engine hang recovery procedure fails at any point in between
disablement and reenablement of the engine there is a back-off
procedure: For gen8 it's possible to back out of the reset handshake by
clearing the reset request bit in the reset control register.
NOTE:
It's possible that some of Ben Widawsky's original per-engine reset patches
from 3 years ago are in this commit but since this work has gone through the
hands of at least 3 people already any kind of ownership tracking has been lost
a long time ago. If you think that you should be on the sob list just let me
know.
* RFCv2: (Chris Wilson / Daniel Vetter)
- Simply use the previously private function i915_gem_reset_ring_status() from
the engine hang recovery path to set active/pending context status. This
replicates the same behaviour as in full GPU reset but for a single,
targetted engine.
- Remove all additional uevents for both full GPU reset and per-engine reset.
Adapted uevent behaviour to the new per-engine hang recovery mode in that it
will only send one uevent regardless of which form of recovery is employed.
If a per-engine reset is attempted first then one uevent will be dispatched.
If that recovery mode fails and the hang is promoted to a full GPU reset no
further uevents will be dispatched at that point.
- Tidied up the TDR context resubmission path in intel_lrc.c . Reduced the
amount of duplication by relying entirely on the normal unqueue function.
Added a new parameter to the unqueue function that takes into consideration
if the unqueue call is for a first-time context submission or a resubmission
and adapts the handling of elsp_submitted accordingly. The reason for
this is that for context resubmission we don't expect any further
interrupts for the submission or the following context completion. A more
elegant way of handling this would be to phase out elsp_submitted
altogether, however that's part of a LRC/execlist cleanup effort that is
happening independently of this patch series. For now we make this change
as simple as possible with as few non-TDR-related side-effects as
possible.
Signed-off-by: Tomas Elf <tomas.elf at intel.com>
Signed-off-by: Ian Lister <ian.lister at intel.com>
Cc: Chris Wilson <chris at chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala at linux.intel.com>
Signed-off-by: Arun Siluvery <arun.siluvery at linux.intel.com>
---
drivers/gpu/drm/i915/i915_dma.c | 18 +
drivers/gpu/drm/i915/i915_drv.c | 206 ++++++++++++
drivers/gpu/drm/i915/i915_drv.h | 58 ++++
drivers/gpu/drm/i915/i915_irq.c | 169 +++++++++-
drivers/gpu/drm/i915/i915_params.c | 19 ++
drivers/gpu/drm/i915/i915_params.h | 2 +
drivers/gpu/drm/i915/i915_reg.h | 2 +
drivers/gpu/drm/i915/intel_lrc.c | 565 +++++++++++++++++++++++++++++++-
drivers/gpu/drm/i915/intel_lrc.h | 14 +
drivers/gpu/drm/i915/intel_lrc_tdr.h | 36 ++
drivers/gpu/drm/i915/intel_ringbuffer.c | 84 ++++-
drivers/gpu/drm/i915/intel_ringbuffer.h | 64 ++++
drivers/gpu/drm/i915/intel_uncore.c | 147 +++++++++
13 files changed, 1358 insertions(+), 26 deletions(-)
create mode 100644 drivers/gpu/drm/i915/intel_lrc_tdr.h
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index 44a896c..c45ec353 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -837,6 +837,22 @@ static void intel_device_info_runtime_init(struct drm_device *dev)
info->has_eu_pg ? "y" : "n");
}
+static void
+i915_hangcheck_init(struct drm_device *dev)
+{
+ int i;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ for (i = 0; i < I915_NUM_RINGS; i++) {
+ struct intel_engine_cs *engine = &dev_priv->ring[i];
+ struct intel_ring_hangcheck *hc = &engine->hangcheck;
+
+ i915_hangcheck_reinit(engine);
+ hc->reset_count = 0;
+ hc->tdr_count = 0;
+ }
+}
+
static void intel_init_dpio(struct drm_i915_private *dev_priv)
{
/*
@@ -1034,6 +1050,8 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
i915_gem_load(dev);
+ i915_hangcheck_init(dev);
+
/* On the 945G/GM, the chipset reports the MSI capability on the
* integrated graphics even though the support isn't actually there
* according to the published specs. It doesn't appear to function
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index f17a2b0..c0ad003 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -34,6 +34,7 @@
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "intel_lrc_tdr.h"
#include <linux/console.h>
#include <linux/module.h>
@@ -571,6 +572,7 @@ static int i915_drm_suspend(struct drm_device *dev)
struct drm_i915_private *dev_priv = dev->dev_private;
pci_power_t opregion_target_state;
int error;
+ int i;
/* ignore lid events during suspend */
mutex_lock(&dev_priv->modeset_restore_lock);
@@ -596,6 +598,16 @@ static int i915_drm_suspend(struct drm_device *dev)
intel_guc_suspend(dev);
+ /*
+ * Clear any pending reset requests. They should be picked up
+ * after resume when new work is submitted
+ */
+ for (i = 0; i < I915_NUM_RINGS; i++)
+ atomic_set(&dev_priv->ring[i].hangcheck.flags, 0);
+
+ atomic_clear_mask(I915_RESET_IN_PROGRESS_FLAG,
+ &dev_priv->gpu_error.reset_counter);
+
intel_suspend_gt_powersave(dev);
/*
@@ -948,6 +960,200 @@ int i915_reset(struct drm_device *dev)
return 0;
}
+/**
+ * i915_reset_engine - reset GPU engine after a hang
+ * @engine: engine to reset
+ *
+ * Reset a specific GPU engine. Useful if a hang is detected. Returns zero on successful
+ * reset or otherwise an error code.
+ *
+ * Procedure is fairly simple:
+ *
+ * - Force engine to idle.
+ *
+ * - Save current head register value and nudge it past the point of the hang in the
+ * ring buffer, which is typically the BB_START instruction of the hung batch buffer,
+ * on to the following instruction.
+ *
+ * - Reset engine.
+ *
+ * - Restore the previously saved, nudged head register value.
+ *
+ * - Re-enable engine to resume running. On gen8 this requires the previously hung
+ * context to be resubmitted to ELSP via the dedicated TDR-execlists interface.
+ *
+ */
+int i915_reset_engine(struct intel_engine_cs *engine)
+{
+ struct drm_device *dev = engine->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_gem_request *current_request = NULL;
+ uint32_t head;
+ bool force_advance = false;
+ int ret = 0;
+ int err_ret = 0;
+
+ WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+ /* Take wake lock to prevent power saving mode */
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+
+ i915_gem_reset_ring_status(dev_priv, engine);
+
+ if (i915.enable_execlists) {
+ enum context_submission_status status =
+ intel_execlists_TDR_get_current_request(engine, NULL);
+
+ /*
+ * If the context submission state in hardware is not
+ * consistent with the the corresponding state in the driver or
+ * if there for some reason is no current context in the
+ * process of being submitted then bail out and try again. Do
+ * not proceed unless we have reliable current context state
+ * information. The reason why this is important is because
+ * per-engine hang recovery relies on context resubmission in
+ * order to force the execution to resume following the hung
+ * batch buffer. If the hardware is not currently running the
+ * same context as the driver thinks is hung then anything can
+ * happen at the point of context resubmission, e.g. unexpected
+ * preemptions or the previously hung context could be
+ * submitted when the hardware is idle which makes no sense.
+ */
+ if (status != CONTEXT_SUBMISSION_STATUS_OK) {
+ ret = -EAGAIN;
+ goto reset_engine_error;
+ }
+ }
+
+ ret = intel_ring_disable(engine);
+ if (ret != 0) {
+ DRM_ERROR("Failed to disable %s\n", engine->name);
+ goto reset_engine_error;
+ }
+
+ if (i915.enable_execlists) {
+ enum context_submission_status status;
+ bool inconsistent;
+
+ status = intel_execlists_TDR_get_current_request(engine,
+ ¤t_request);
+
+ inconsistent = (status != CONTEXT_SUBMISSION_STATUS_OK);
+ if (inconsistent) {
+ /*
+ * If we somehow have reached this point with
+ * an inconsistent context submission status then
+ * back out of the previously requested reset and
+ * retry later.
+ */
+ WARN(inconsistent,
+ "Inconsistent context status on %s: %u\n",
+ engine->name, status);
+
+ ret = -EAGAIN;
+ goto reenable_reset_engine_error;
+ }
+ }
+
+ /* Sample the current ring head position */
+ head = I915_READ_HEAD(engine) & HEAD_ADDR;
+
+ if (head == engine->hangcheck.last_head) {
+ /*
+ * The engine has not advanced since the last
+ * time it hung so force it to advance to the
+ * next QWORD. In most cases the engine head
+ * pointer will automatically advance to the
+ * next instruction as soon as it has read the
+ * current instruction, without waiting for it
+ * to complete. This seems to be the default
+ * behaviour, however an MBOX wait inserted
+ * directly to the VCS/BCS engines does not behave
+ * in the same way, instead the head pointer
+ * will still be pointing at the MBOX instruction
+ * until it completes.
+ */
+ force_advance = true;
+ }
+
+ engine->hangcheck.last_head = head;
+
+ ret = intel_ring_save(engine, current_request, force_advance);
+ if (ret) {
+ DRM_ERROR("Failed to save %s engine state\n", engine->name);
+ goto reenable_reset_engine_error;
+ }
+
+ ret = intel_gpu_engine_reset(engine);
+ if (ret) {
+ DRM_ERROR("Failed to reset %s\n", engine->name);
+ goto reenable_reset_engine_error;
+ }
+
+ ret = intel_ring_restore(engine, current_request);
+ if (ret) {
+ DRM_ERROR("Failed to restore %s engine state\n", engine->name);
+ goto reenable_reset_engine_error;
+ }
+
+ /* Correct driver state */
+ intel_gpu_engine_reset_resample(engine, current_request);
+
+ /*
+ * Reenable engine
+ *
+ * In execlist mode on gen8+ this is implicit by simply resubmitting
+ * the previously hung context. In ring buffer submission mode on gen7
+ * and earlier we need to actively turn on the engine first.
+ */
+ if (i915.enable_execlists)
+ intel_execlists_TDR_context_resubmission(engine);
+ else
+ ret = intel_ring_enable(engine);
+
+ if (ret) {
+ DRM_ERROR("Failed to enable %s again after reset\n",
+ engine->name);
+
+ goto reset_engine_error;
+ }
+
+ /* Clear reset flags to allow future hangchecks */
+ atomic_set(&engine->hangcheck.flags, 0);
+
+ /* Wake up anything waiting on this engine's queue */
+ wake_up_all(&engine->irq_queue);
+
+ if (i915.enable_execlists && current_request)
+ i915_gem_request_unreference(current_request);
+
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ return ret;
+
+reenable_reset_engine_error:
+
+ err_ret = intel_ring_enable(engine);
+ if (err_ret)
+ DRM_ERROR("Failed to reenable %s following error during reset (%d)\n",
+ engine->name, err_ret);
+
+reset_engine_error:
+
+ /* Clear reset flags to allow future hangchecks */
+ atomic_set(&engine->hangcheck.flags, 0);
+
+ /* Wake up anything waiting on this engine's queue */
+ wake_up_all(&engine->irq_queue);
+
+ if (i915.enable_execlists && current_request)
+ i915_gem_request_unreference(current_request);
+
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ return ret;
+}
+
static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct intel_device_info *intel_info =
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 703a320..e866f14 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2432,6 +2432,48 @@ struct drm_i915_cmd_table {
int count;
};
+/*
+ * Context submission status
+ *
+ * CONTEXT_SUBMISSION_STATUS_OK:
+ * Context submitted to ELSP and state of execlist queue is the same as
+ * the state of EXECLIST_STATUS register. Software and hardware states
+ * are consistent and can be trusted.
+ *
+ * CONTEXT_SUBMISSION_STATUS_INCONSISTENT:
+ * Context has been submitted to the execlist queue but the state of the
+ * EXECLIST_STATUS register is different from the execlist queue state.
+ * This could mean any of the following:
+ *
+ * 1. The context is in the head position of the execlist queue
+ * but has not yet been submitted to ELSP.
+ *
+ * 2. The hardware just recently completed the context but the
+ * context is pending removal from the execlist queue.
+ *
+ * 3. The driver has lost a context state transition interrupt.
+ * Typically what this means is that hardware has completed and
+ * is now idle but the driver thinks the hardware is still
+ * busy.
+ *
+ * Overall what this means is that the context submission status is
+ * currently in transition and cannot be trusted until it settles down.
+ *
+ * CONTEXT_SUBMISSION_STATUS_NONE_SUBMITTED:
+ * No context submitted to the execlist queue and the EXECLIST_STATUS
+ * register shows no context being processed.
+ *
+ * CONTEXT_SUBMISSION_STATUS_NONE_UNDEFINED:
+ * Initial state before submission status has been determined.
+ *
+ */
+enum context_submission_status {
+ CONTEXT_SUBMISSION_STATUS_OK = 0,
+ CONTEXT_SUBMISSION_STATUS_INCONSISTENT,
+ CONTEXT_SUBMISSION_STATUS_NONE_SUBMITTED,
+ CONTEXT_SUBMISSION_STATUS_UNDEFINED
+};
+
/* Note that the (struct drm_i915_private *) cast is just to shut up gcc. */
#define __I915__(p) ({ \
struct drm_i915_private *__p; \
@@ -2690,8 +2732,12 @@ extern long i915_compat_ioctl(struct file *filp, unsigned int cmd,
unsigned long arg);
#endif
extern int intel_gpu_reset(struct drm_device *dev);
+extern int intel_gpu_engine_reset(struct intel_engine_cs *engine);
+extern int intel_request_gpu_engine_reset(struct intel_engine_cs *engine);
+extern int intel_unrequest_gpu_engine_reset(struct intel_engine_cs *engine);
extern bool intel_has_gpu_reset(struct drm_device *dev);
extern int i915_reset(struct drm_device *dev);
+extern int i915_reset_engine(struct intel_engine_cs *engine);
extern unsigned long i915_chipset_val(struct drm_i915_private *dev_priv);
extern unsigned long i915_mch_val(struct drm_i915_private *dev_priv);
extern unsigned long i915_gfx_val(struct drm_i915_private *dev_priv);
@@ -2704,6 +2750,18 @@ void intel_hpd_init(struct drm_i915_private *dev_priv);
void intel_hpd_init_work(struct drm_i915_private *dev_priv);
void intel_hpd_cancel_work(struct drm_i915_private *dev_priv);
bool intel_hpd_pin_to_port(enum hpd_pin pin, enum port *port);
+static inline void i915_hangcheck_reinit(struct intel_engine_cs *engine)
+{
+ struct intel_ring_hangcheck *hc = &engine->hangcheck;
+
+ hc->acthd = 0;
+ hc->max_acthd = 0;
+ hc->seqno = 0;
+ hc->score = 0;
+ hc->action = HANGCHECK_IDLE;
+ hc->deadlock = 0;
+}
+
/* i915_irq.c */
void i915_queue_hangcheck(struct drm_device *dev);
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index f04d799..6a0ec37 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -2470,10 +2470,70 @@ static void i915_reset_and_wakeup(struct drm_device *dev)
char *error_event[] = { I915_ERROR_UEVENT "=1", NULL };
char *reset_event[] = { I915_RESET_UEVENT "=1", NULL };
char *reset_done_event[] = { I915_ERROR_UEVENT "=0", NULL };
- int ret;
+ bool reset_complete = false;
+ struct intel_engine_cs *ring;
+ int ret = 0;
+ int i;
+
+ mutex_lock(&dev->struct_mutex);
kobject_uevent_env(&dev->primary->kdev->kobj, KOBJ_CHANGE, error_event);
+ for_each_ring(ring, dev_priv, i) {
+
+ /*
+ * Skip further individual engine reset requests if full GPU
+ * reset requested.
+ */
+ if (i915_reset_in_progress(error))
+ break;
+
+ if (atomic_read(&ring->hangcheck.flags) &
+ I915_ENGINE_RESET_IN_PROGRESS) {
+
+ if (!reset_complete)
+ kobject_uevent_env(&dev->primary->kdev->kobj,
+ KOBJ_CHANGE,
+ reset_event);
+
+ reset_complete = true;
+
+ ret = i915_reset_engine(ring);
+
+ /*
+ * Execlist mode only:
+ *
+ * -EAGAIN means that between detecting a hang (and
+ * also determining that the currently submitted
+ * context is stable and valid) and trying to recover
+ * from the hang the current context changed state.
+ * This means that we are probably not completely hung
+ * after all. Just fail and retry by exiting all the
+ * way back and wait for the next hang detection. If we
+ * have a true hang on our hands then we will detect it
+ * again, otherwise we will continue like nothing
+ * happened.
+ */
+ if (ret == -EAGAIN) {
+ DRM_ERROR("Reset of %s aborted due to " \
+ "change in context submission " \
+ "state - retrying!", ring->name);
+ ret = 0;
+ }
+
+ if (ret) {
+ DRM_ERROR("Reset of %s failed! (%d)", ring->name, ret);
+
+ atomic_or(I915_RESET_IN_PROGRESS_FLAG,
+ &dev_priv->gpu_error.reset_counter);
+ break;
+ }
+ }
+ }
+
+ /* The full GPU reset will grab the struct_mutex when it needs it */
+ mutex_unlock(&dev->struct_mutex);
+
/*
* Note that there's only one work item which does gpu resets, so we
* need not worry about concurrent gpu resets potentially incrementing
@@ -2486,8 +2546,13 @@ static void i915_reset_and_wakeup(struct drm_device *dev)
*/
if (i915_reset_in_progress(error) && !i915_terminally_wedged(error)) {
DRM_DEBUG_DRIVER("resetting chip\n");
- kobject_uevent_env(&dev->primary->kdev->kobj, KOBJ_CHANGE,
- reset_event);
+
+ if (!reset_complete)
+ kobject_uevent_env(&dev->primary->kdev->kobj,
+ KOBJ_CHANGE,
+ reset_event);
+
+ reset_complete = true;
/*
* In most cases it's guaranteed that we get here with an RPM
@@ -2520,23 +2585,36 @@ static void i915_reset_and_wakeup(struct drm_device *dev)
*
* Since unlock operations are a one-sided barrier only,
* we need to insert a barrier here to order any seqno
- * updates before
- * the counter increment.
+ * updates before the counter increment.
+ *
+ * The increment clears I915_RESET_IN_PROGRESS_FLAG.
*/
smp_mb__before_atomic();
atomic_inc(&dev_priv->gpu_error.reset_counter);
- kobject_uevent_env(&dev->primary->kdev->kobj,
- KOBJ_CHANGE, reset_done_event);
+ /*
+ * If any per-engine resets were promoted to full GPU
+ * reset don't forget to clear those reset flags.
+ */
+ for_each_ring(ring, dev_priv, i)
+ atomic_set(&ring->hangcheck.flags, 0);
} else {
+ /* Terminal wedge condition */
+ WARN(1, "i915_reset failed, declaring GPU as wedged!\n");
atomic_or(I915_WEDGED, &error->reset_counter);
}
+ }
- /*
- * Note: The wake_up also serves as a memory barrier so that
- * waiters see the update value of the reset counter atomic_t.
- */
+ /*
+ * Note: The wake_up also serves as a memory barrier so that
+ * waiters see the update value of the reset counter atomic_t.
+ */
+ if (reset_complete) {
i915_error_wake_up(dev_priv, true);
+
+ if (ret == 0)
+ kobject_uevent_env(&dev->primary->kdev->kobj,
+ KOBJ_CHANGE, reset_done_event);
}
}
@@ -2649,6 +2727,14 @@ void i915_handle_error(struct drm_device *dev, bool wedged,
va_list args;
char error_msg[80];
+ struct intel_engine_cs *engine;
+
+ /*
+ * NB: Placeholder until the hang checker supports
+ * per-engine hang detection.
+ */
+ u32 engine_mask = 0;
+
va_start(args, fmt);
vscnprintf(error_msg, sizeof(error_msg), fmt, args);
va_end(args);
@@ -2657,8 +2743,65 @@ void i915_handle_error(struct drm_device *dev, bool wedged,
i915_report_and_clear_eir(dev);
if (wedged) {
- atomic_or(I915_RESET_IN_PROGRESS_FLAG,
- &dev_priv->gpu_error.reset_counter);
+ /*
+ * Defer to full GPU reset if any of the following is true:
+ * 0. Engine reset disabled.
+ * 1. The caller did not ask for per-engine reset.
+ * 2. The hardware does not support it (pre-gen7).
+ * 3. We already tried per-engine reset recently.
+ */
+ bool full_reset = true;
+
+ if (!i915.enable_engine_reset) {
+ DRM_INFO("Engine reset disabled: Using full GPU reset.\n");
+ engine_mask = 0x0;
+ }
+
+ /*
+ * TBD: We currently only support per-engine reset for gen8+.
+ * Implement support for gen7.
+ */
+ if (engine_mask && (INTEL_INFO(dev)->gen >= 8)) {
+ u32 i;
+
+ for_each_ring(engine, dev_priv, i) {
+ u32 now, last_engine_reset_timediff;
+
+ if (!(intel_ring_flag(engine) & engine_mask))
+ continue;
+
+ /* Measure the time since this engine was last reset */
+ now = get_seconds();
+ last_engine_reset_timediff =
+ now - engine->hangcheck.last_engine_reset_time;
+
+ full_reset = last_engine_reset_timediff <
+ i915.gpu_reset_promotion_time;
+
+ engine->hangcheck.last_engine_reset_time = now;
+
+ /*
+ * This engine was not reset too recently - go ahead
+ * with engine reset instead of falling back to full
+ * GPU reset.
+ *
+ * Flag that we want to try and reset this engine.
+ * This can still be overridden by a global
+ * reset e.g. if per-engine reset fails.
+ */
+ if (!full_reset)
+ atomic_or(I915_ENGINE_RESET_IN_PROGRESS,
+ &engine->hangcheck.flags);
+ else
+ break;
+
+ } /* for_each_ring */
+ }
+
+ if (full_reset) {
+ atomic_or(I915_RESET_IN_PROGRESS_FLAG,
+ &dev_priv->gpu_error.reset_counter);
+ }
/*
* Wakeup waiting processes so that the reset function
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index 8d90c25..5cf9c11 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -37,6 +37,8 @@ struct i915_params i915 __read_mostly = {
.enable_fbc = -1,
.enable_execlists = -1,
.enable_hangcheck = true,
+ .enable_engine_reset = false,
+ .gpu_reset_promotion_time = 10,
.enable_ppgtt = -1,
.enable_psr = 0,
.preliminary_hw_support = IS_ENABLED(CONFIG_DRM_I915_PRELIMINARY_HW_SUPPORT),
@@ -116,6 +118,23 @@ MODULE_PARM_DESC(enable_hangcheck,
"WARNING: Disabling this can cause system wide hangs. "
"(default: true)");
+module_param_named_unsafe(enable_engine_reset, i915.enable_engine_reset, bool, 0644);
+MODULE_PARM_DESC(enable_engine_reset,
+ "Enable GPU engine hang recovery mode. Used as a soft, low-impact form "
+ "of hang recovery that targets individual GPU engines rather than the "
+ "entire GPU"
+ "(default: false)");
+
+module_param_named(gpu_reset_promotion_time,
+ i915.gpu_reset_promotion_time, int, 0644);
+MODULE_PARM_DESC(gpu_reset_promotion_time,
+ "Catch excessive engine resets. Each engine maintains a "
+ "timestamp of the last time it was reset. If it hangs again "
+ "within this period then fall back to full GPU reset to try and"
+ " recover from the hang. Only applicable if enable_engine_reset "
+ "is enabled."
+ "default=10 seconds");
+
module_param_named_unsafe(enable_ppgtt, i915.enable_ppgtt, int, 0400);
MODULE_PARM_DESC(enable_ppgtt,
"Override PPGTT usage. "
diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
index 5299290..60f3d23 100644
--- a/drivers/gpu/drm/i915/i915_params.h
+++ b/drivers/gpu/drm/i915/i915_params.h
@@ -49,8 +49,10 @@ struct i915_params {
int use_mmio_flip;
int mmio_debug;
int edp_vswing;
+ unsigned int gpu_reset_promotion_time;
/* leave bools at the end to not create holes */
bool enable_hangcheck;
+ bool enable_engine_reset;
bool fastboot;
bool prefault_disable;
bool load_detect_test;
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index 0a98889..3fc5d75 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -164,6 +164,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN6_GRDOM_RENDER (1 << 1)
#define GEN6_GRDOM_MEDIA (1 << 2)
#define GEN6_GRDOM_BLT (1 << 3)
+#define GEN6_GRDOM_VECS (1 << 4)
+#define GEN8_GRDOM_MEDIA2 (1 << 7)
#define RING_PP_DIR_BASE(ring) _MMIO((ring)->mmio_base+0x228)
#define RING_PP_DIR_BASE_READ(ring) _MMIO((ring)->mmio_base+0x518)
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index ab344e0..fcec476 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -136,6 +136,7 @@
#include <drm/i915_drm.h>
#include "i915_drv.h"
#include "intel_mocs.h"
+#include "intel_lrc_tdr.h"
#define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
#define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -325,7 +326,8 @@ uint64_t intel_lr_context_descriptor(struct intel_context *ctx,
}
static void execlists_elsp_write(struct drm_i915_gem_request *rq0,
- struct drm_i915_gem_request *rq1)
+ struct drm_i915_gem_request *rq1,
+ bool tdr_resubmission)
{
struct intel_engine_cs *ring = rq0->ring;
@@ -335,13 +337,17 @@ static void execlists_elsp_write(struct drm_i915_gem_request *rq0,
if (rq1) {
desc[1] = intel_lr_context_descriptor(rq1->ctx, rq1->ring);
- rq1->elsp_submitted++;
+
+ if (!tdr_resubmission)
+ rq1->elsp_submitted++;
} else {
desc[1] = 0;
}
desc[0] = intel_lr_context_descriptor(rq0->ctx, rq0->ring);
- rq0->elsp_submitted++;
+
+ if (!tdr_resubmission)
+ rq0->elsp_submitted++;
/* You must always write both descriptors in the order below. */
spin_lock(&dev_priv->uncore.lock);
@@ -359,6 +365,182 @@ static void execlists_elsp_write(struct drm_i915_gem_request *rq0,
spin_unlock(&dev_priv->uncore.lock);
}
+/**
+ * execlist_get_context_reg_page() - Get memory page for context object
+ * @engine: engine
+ * @ctx: context running on engine
+ * @page: returned page
+ *
+ * Return: 0 if successful, otherwise propagates error codes.
+ */
+static inline int execlist_get_context_reg_page(struct intel_engine_cs *engine,
+ struct intel_context *ctx,
+ struct page **page)
+{
+ struct drm_i915_gem_object *ctx_obj;
+
+ if (!page)
+ return -EINVAL;
+
+ if (!ctx)
+ ctx = engine->default_context;
+
+ ctx_obj = ctx->engine[engine->id].state;
+
+ if (WARN(!ctx_obj, "Context object not set up!\n"))
+ return -EINVAL;
+
+ WARN(!i915_gem_obj_is_pinned(ctx_obj),
+ "Context object is not pinned!\n");
+
+ *page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN);
+
+ if (WARN(!*page, "Context object page could not be resolved!\n"))
+ return -EINVAL;
+
+ return 0;
+}
+
+/**
+ * execlist_write_context_reg() - Write value to Context register
+ * @engine: Engine
+ * @ctx: Context running on engine
+ * @ctx_reg: Index into context image pointing to register location
+ * @mmio_reg_addr: MMIO register address
+ * @val: Value to be written
+ * @mmio_reg_name_str: Designated register name
+ *
+ * Return: 0 if successful, otherwise propagates error codes.
+ */
+static inline int execlists_write_context_reg(struct intel_engine_cs *engine,
+ struct intel_context *ctx,
+ u32 ctx_reg,
+ i915_reg_t mmio_reg,
+ u32 val,
+ const char *mmio_reg_name_str)
+{
+ struct page *page = NULL;
+ uint32_t *reg_state;
+
+ int ret = execlist_get_context_reg_page(engine, ctx, &page);
+ if (WARN(ret, "[write %s:%u] Failed to get context memory page for %s!\n",
+ mmio_reg_name_str, (unsigned int) mmio_reg.reg, engine->name)) {
+ return ret;
+ }
+
+ reg_state = kmap_atomic(page);
+
+ WARN(reg_state[ctx_reg] != mmio_reg.reg,
+ "[write %s:%u]: Context reg addr (%x) != MMIO reg addr (%x)!\n",
+ mmio_reg_name_str,
+ (unsigned int) mmio_reg.reg,
+ (unsigned int) reg_state[ctx_reg],
+ (unsigned int) mmio_reg.reg);
+
+ reg_state[ctx_reg+1] = val;
+ kunmap_atomic(reg_state);
+
+ return ret;
+}
+
+/**
+ * execlist_read_context_reg() - Read value from Context register
+ * @engine: Engine
+ * @ctx: Context running on engine
+ * @ctx_reg: Index into context image pointing to register location
+ * @mmio_reg: MMIO register struct
+ * @val: Output parameter returning register value
+ * @mmio_reg_name_str: Designated register name
+ *
+ * Return: 0 if successful, otherwise propagates error codes.
+ */
+static inline int execlists_read_context_reg(struct intel_engine_cs *engine,
+ struct intel_context *ctx,
+ u32 ctx_reg,
+ i915_reg_t mmio_reg,
+ u32 *val,
+ const char *mmio_reg_name_str)
+{
+ struct page *page = NULL;
+ uint32_t *reg_state;
+ int ret = 0;
+
+ if (!val)
+ return -EINVAL;
+
+ ret = execlist_get_context_reg_page(engine, ctx, &page);
+ if (WARN(ret, "[read %s:%u] Failed to get context memory page for %s!\n",
+ mmio_reg_name_str, (unsigned int) mmio_reg.reg, engine->name)) {
+ return ret;
+ }
+
+ reg_state = kmap_atomic(page);
+
+ WARN(reg_state[ctx_reg] != mmio_reg.reg,
+ "[read %s:%u]: Context reg addr (%x) != MMIO reg addr (%x)!\n",
+ mmio_reg_name_str,
+ (unsigned int) ctx_reg,
+ (unsigned int) reg_state[ctx_reg],
+ (unsigned int) mmio_reg.reg);
+
+ *val = reg_state[ctx_reg+1];
+ kunmap_atomic(reg_state);
+
+ return ret;
+ }
+
+/*
+ * Generic macros for generating function implementation for context register
+ * read/write functions.
+ *
+ * Macro parameters
+ * ----------------
+ * reg_name: Designated name of context register (e.g. tail, head, buffer_ctl)
+ *
+ * reg_def: Context register macro definition (e.g. CTX_RING_TAIL)
+ *
+ * mmio_reg_def: Name of macro function used to determine the address
+ * of the corresponding MMIO register (e.g. RING_TAIL, RING_HEAD).
+ * This macro function is assumed to be defined on the form of:
+ *
+ * #define mmio_reg_def(base) (base+register_offset)
+ *
+ * Where "base" is the MMIO base address of the respective ring
+ * and "register_offset" is the offset relative to "base".
+ *
+ * Function parameters
+ * -------------------
+ * engine: The engine that the context is running on
+ * ctx: The context of the register that is to be accessed
+ * reg_name: Value to be written/read to/from the register.
+ */
+#define INTEL_EXECLISTS_WRITE_REG(reg_name, reg_def, mmio_reg_def) \
+ int intel_execlists_write_##reg_name(struct intel_engine_cs *engine, \
+ struct intel_context *ctx, \
+ u32 reg_name) \
+{ \
+ return execlists_write_context_reg(engine, ctx, (reg_def), \
+ mmio_reg_def(engine->mmio_base), (reg_name), \
+ (#reg_name)); \
+}
+
+#define INTEL_EXECLISTS_READ_REG(reg_name, reg_def, mmio_reg_def) \
+ int intel_execlists_read_##reg_name(struct intel_engine_cs *engine, \
+ struct intel_context *ctx, \
+ u32 *reg_name) \
+{ \
+ return execlists_read_context_reg(engine, ctx, (reg_def), \
+ mmio_reg_def(engine->mmio_base), (reg_name), \
+ (#reg_name)); \
+}
+
+INTEL_EXECLISTS_READ_REG(tail, CTX_RING_TAIL, RING_TAIL)
+INTEL_EXECLISTS_WRITE_REG(head, CTX_RING_HEAD, RING_HEAD)
+INTEL_EXECLISTS_READ_REG(head, CTX_RING_HEAD, RING_HEAD)
+
+#undef INTEL_EXECLISTS_READ_REG
+#undef INTEL_EXECLISTS_WRITE_REG
+
static int execlists_update_context(struct drm_i915_gem_request *rq)
{
struct intel_engine_cs *ring = rq->ring;
@@ -396,17 +578,18 @@ static int execlists_update_context(struct drm_i915_gem_request *rq)
}
static void execlists_submit_requests(struct drm_i915_gem_request *rq0,
- struct drm_i915_gem_request *rq1)
+ struct drm_i915_gem_request *rq1,
+ bool tdr_resubmission)
{
execlists_update_context(rq0);
if (rq1)
execlists_update_context(rq1);
- execlists_elsp_write(rq0, rq1);
+ execlists_elsp_write(rq0, rq1, tdr_resubmission);
}
-static void execlists_context_unqueue(struct intel_engine_cs *ring)
+static void execlists_context_unqueue(struct intel_engine_cs *ring, bool tdr_resubmission)
{
struct drm_i915_gem_request *req0 = NULL, *req1 = NULL;
struct drm_i915_gem_request *cursor = NULL, *tmp = NULL;
@@ -440,6 +623,16 @@ static void execlists_context_unqueue(struct intel_engine_cs *ring)
}
}
+ /*
+ * Only do TDR resubmission of the second head request if it's already
+ * been submitted. The intention is to restore the original submission
+ * state from the situation when the hang originally happened. If it
+ * was never submitted we don't want to submit it for the first time at
+ * this point
+ */
+ if (tdr_resubmission && req1 && !req1->elsp_submitted)
+ req1 = NULL;
+
if (IS_GEN8(ring->dev) || IS_GEN9(ring->dev)) {
/*
* WaIdleLiteRestore: make sure we never cause a lite
@@ -460,9 +653,32 @@ static void execlists_context_unqueue(struct intel_engine_cs *ring)
}
}
- WARN_ON(req1 && req1->elsp_submitted);
+ WARN_ON(req1 && req1->elsp_submitted && !tdr_resubmission);
- execlists_submit_requests(req0, req1);
+ execlists_submit_requests(req0, req1, tdr_resubmission);
+}
+
+/**
+ * intel_execlists_TDR_context_resubmission() - ELSP context resubmission
+ * @ring: engine to do resubmission for.
+ *
+ * Context submission mechanism exclusively used by TDR that bypasses the
+ * execlist queue. This is necessary since at the point of TDR hang recovery
+ * the hardware will be hung and resubmitting a fixed context (the context that
+ * the TDR has identified as hung and fixed up in order to move past the
+ * blocking batch buffer) to a hung execlist queue will lock up the TDR.
+ * Instead, opt for direct ELSP submission without depending on the rest of the
+ * driver.
+ */
+void intel_execlists_TDR_context_resubmission(struct intel_engine_cs *ring)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ring->execlist_lock, flags);
+ WARN_ON(list_empty(&ring->execlist_queue));
+
+ execlists_context_unqueue(ring, true);
+ spin_unlock_irqrestore(&ring->execlist_lock, flags);
}
static bool execlists_check_remove_request(struct intel_engine_cs *ring,
@@ -560,9 +776,9 @@ void intel_lrc_irq_handler(struct intel_engine_cs *ring)
/* Prevent a ctx to preempt itself */
if ((status & GEN8_CTX_STATUS_ACTIVE_IDLE) &&
(submit_contexts != 0))
- execlists_context_unqueue(ring);
+ execlists_context_unqueue(ring, false);
} else if (submit_contexts != 0) {
- execlists_context_unqueue(ring);
+ execlists_context_unqueue(ring, false);
}
spin_unlock(&ring->execlist_lock);
@@ -613,7 +829,7 @@ static int execlists_context_queue(struct drm_i915_gem_request *request)
list_add_tail(&request->execlist_link, &ring->execlist_queue);
if (num_elements == 0)
- execlists_context_unqueue(ring);
+ execlists_context_unqueue(ring, false);
spin_unlock_irq(&ring->execlist_lock);
@@ -1536,7 +1752,7 @@ static int gen8_init_common_ring(struct intel_engine_cs *ring)
ring->next_context_status_buffer = next_context_status_buffer_hw;
DRM_DEBUG_DRIVER("Execlists enabled for %s\n", ring->name);
- memset(&ring->hangcheck, 0, sizeof(ring->hangcheck));
+ i915_hangcheck_reinit(ring);
return 0;
}
@@ -1888,6 +2104,187 @@ out:
return ret;
}
+static int
+gen8_ring_disable(struct intel_engine_cs *ring)
+{
+ intel_request_gpu_engine_reset(ring);
+ return 0;
+}
+
+static int
+gen8_ring_enable(struct intel_engine_cs *ring)
+{
+ intel_unrequest_gpu_engine_reset(ring);
+ return 0;
+}
+
+/**
+ * gen8_ring_save() - save minimum engine state
+ * @ring: engine whose state is to be saved
+ * @req: request containing the context currently running on engine
+ * @force_advance: indicates whether or not we should nudge the head
+ * forward or not
+ *
+ * Saves the head MMIO register to scratch memory while engine is reset and
+ * reinitialized. Before saving the head register we nudge the head position to
+ * be correctly aligned with a QWORD boundary, which brings it up to the next
+ * presumably valid instruction. Typically, at the point of hang recovery the
+ * head register will be pointing to the last DWORD of the BB_START
+ * instruction, which is followed by a padding MI_NOOP inserted by the
+ * driver.
+ *
+ * Returns:
+ * 0 if ok, otherwise propagates error codes.
+ */
+static int
+gen8_ring_save(struct intel_engine_cs *ring, struct drm_i915_gem_request *req,
+ bool force_advance)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct intel_ringbuffer *ringbuf = NULL;
+ struct intel_context *ctx;
+ int ret = 0;
+ int clamp_to_tail = 0;
+ uint32_t head;
+ uint32_t tail;
+ uint32_t head_addr;
+ uint32_t tail_addr;
+
+ if (WARN_ON(!req))
+ return -EINVAL;
+
+ ctx = req->ctx;
+ ringbuf = ctx->engine[ring->id].ringbuf;
+
+ /*
+ * Read head from MMIO register since it contains the
+ * most up to date value of head at this point.
+ */
+ head = I915_READ_HEAD(ring);
+
+ /*
+ * Read tail from the context because the execlist queue
+ * updates the tail value there first during submission.
+ * The MMIO tail register is not updated until the actual
+ * ring submission completes.
+ */
+ ret = I915_READ_TAIL_CTX(ring, ctx, tail);
+ if (ret)
+ return ret;
+
+ /*
+ * head_addr and tail_addr are the head and tail values
+ * excluding ring wrapping information and aligned to DWORD
+ * boundary
+ */
+ head_addr = head & HEAD_ADDR;
+ tail_addr = tail & TAIL_ADDR;
+
+ /*
+ * The head must always chase the tail.
+ * If the tail is beyond the head then do not allow
+ * the head to overtake it. If the tail is less than
+ * the head then the tail has already wrapped and
+ * there is no problem in advancing the head or even
+ * wrapping the head back to 0 as worst case it will
+ * become equal to tail
+ */
+ if (head_addr <= tail_addr)
+ clamp_to_tail = 1;
+
+ if (force_advance) {
+
+ /* Force head pointer to next QWORD boundary */
+ head_addr &= ~0x7;
+ head_addr += 8;
+
+ } else if (head & 0x7) {
+
+ /* Ensure head pointer is pointing to a QWORD boundary */
+ head += 0x7;
+ head &= ~0x7;
+ head_addr = head;
+ }
+
+ if (clamp_to_tail && (head_addr > tail_addr)) {
+ head_addr = tail_addr;
+ } else if (head_addr >= ringbuf->size) {
+ /* Wrap head back to start if it exceeds ring size */
+ head_addr = 0;
+ }
+
+ head &= ~HEAD_ADDR;
+ head |= (head_addr & HEAD_ADDR);
+ ring->saved_head = head;
+
+ return 0;
+}
+
+
+/**
+ * gen8_ring_restore() - restore previously saved engine state
+ * @ring: engine whose state is to be restored
+ * @req: request containing the context currently running on engine
+ *
+ * Reinitializes engine and restores the previously saved engine state.
+ * See: gen8_ring_save()
+ *
+ * Returns:
+ * 0 if ok, otherwise propagates error codes.
+ */
+static int
+gen8_ring_restore(struct intel_engine_cs *ring, struct drm_i915_gem_request *req)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct intel_context *ctx;
+
+ if (WARN_ON(!req))
+ return -EINVAL;
+
+ ctx = req->ctx;
+
+ /* Re-initialize ring */
+ if (ring->init_hw) {
+ int ret = ring->init_hw(ring);
+ if (ret != 0) {
+ DRM_ERROR("Failed to re-initialize %s\n",
+ ring->name);
+ return ret;
+ }
+ } else {
+ DRM_ERROR("ring init function pointer not set up\n");
+ return -EINVAL;
+ }
+
+ if (ring->id == RCS) {
+ /*
+ * These register reinitializations are only located here
+ * temporarily until they are moved out of the
+ * init_clock_gating function to some function we can
+ * call from here.
+ */
+
+ /* WaVSRefCountFullforceMissDisable:chv */
+ /* WaDSRefCountFullforceMissDisable:chv */
+ I915_WRITE(GEN7_FF_THREAD_MODE,
+ I915_READ(GEN7_FF_THREAD_MODE) &
+ ~(GEN8_FF_DS_REF_CNT_FFME | GEN7_FF_VS_REF_CNT_FFME));
+
+ I915_WRITE(_3D_CHICKEN3,
+ _3D_CHICKEN_SDE_LIMIT_FIFO_POLY_DEPTH(2));
+
+ /* WaSwitchSolVfFArbitrationPriority:bdw */
+ I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) | HSW_ECOCHK_ARB_PRIO_SOL);
+ }
+
+ /* Restore head */
+
+ I915_WRITE_HEAD(ring, ring->saved_head);
+ I915_WRITE_HEAD_CTX(ring, ctx, ring->saved_head);
+
+ return 0;
+}
+
static int gen8_init_rcs_context(struct drm_i915_gem_request *req)
{
int ret;
@@ -2021,6 +2418,10 @@ static int logical_render_ring_init(struct drm_device *dev)
ring->irq_get = gen8_logical_ring_get_irq;
ring->irq_put = gen8_logical_ring_put_irq;
ring->emit_bb_start = gen8_emit_bb_start;
+ ring->enable = gen8_ring_enable;
+ ring->disable = gen8_ring_disable;
+ ring->save = gen8_ring_save;
+ ring->restore = gen8_ring_restore;
ring->dev = dev;
@@ -2073,6 +2474,10 @@ static int logical_bsd_ring_init(struct drm_device *dev)
ring->irq_get = gen8_logical_ring_get_irq;
ring->irq_put = gen8_logical_ring_put_irq;
ring->emit_bb_start = gen8_emit_bb_start;
+ ring->enable = gen8_ring_enable;
+ ring->disable = gen8_ring_disable;
+ ring->save = gen8_ring_save;
+ ring->restore = gen8_ring_restore;
return logical_ring_init(dev, ring);
}
@@ -2098,6 +2503,10 @@ static int logical_bsd2_ring_init(struct drm_device *dev)
ring->irq_get = gen8_logical_ring_get_irq;
ring->irq_put = gen8_logical_ring_put_irq;
ring->emit_bb_start = gen8_emit_bb_start;
+ ring->enable = gen8_ring_enable;
+ ring->disable = gen8_ring_disable;
+ ring->save = gen8_ring_save;
+ ring->restore = gen8_ring_restore;
return logical_ring_init(dev, ring);
}
@@ -2128,6 +2537,10 @@ static int logical_blt_ring_init(struct drm_device *dev)
ring->irq_get = gen8_logical_ring_get_irq;
ring->irq_put = gen8_logical_ring_put_irq;
ring->emit_bb_start = gen8_emit_bb_start;
+ ring->enable = gen8_ring_enable;
+ ring->disable = gen8_ring_disable;
+ ring->save = gen8_ring_save;
+ ring->restore = gen8_ring_restore;
return logical_ring_init(dev, ring);
}
@@ -2158,6 +2571,10 @@ static int logical_vebox_ring_init(struct drm_device *dev)
ring->irq_get = gen8_logical_ring_get_irq;
ring->irq_put = gen8_logical_ring_put_irq;
ring->emit_bb_start = gen8_emit_bb_start;
+ ring->enable = gen8_ring_enable;
+ ring->disable = gen8_ring_disable;
+ ring->save = gen8_ring_save;
+ ring->restore = gen8_ring_restore;
return logical_ring_init(dev, ring);
}
@@ -2587,3 +3004,127 @@ void intel_lr_context_reset(struct drm_device *dev,
ringbuf->tail = 0;
}
}
+
+/**
+ * intel_execlists_TDR_get_current_request() - return request currently
+ * processed by engine
+ *
+ * @ring: Engine currently running context to be returned.
+ *
+ * @req: Output parameter containing the current request (the request at the
+ * head of execlist queue corresponding to the given ring). May be NULL
+ * if no request has been submitted to the execlist queue of this
+ * engine. If the req parameter passed in to the function is not NULL
+ * and a request is found and returned the request is referenced before
+ * it is returned. It is the responsibility of the caller to dereference
+ * it at the end of its life cycle.
+ *
+ * Return:
+ * CONTEXT_SUBMISSION_STATUS_OK if request is found to be submitted and its
+ * context is currently running on engine.
+ *
+ * CONTEXT_SUBMISSION_STATUS_INCONSISTENT if request is found to be submitted
+ * but its context is not in a state that is consistent with current
+ * hardware state for the given engine. This has been observed in three cases:
+ *
+ * 1. Before the engine has switched to this context after it has
+ * been submitted to the execlist queue.
+ *
+ * 2. After the engine has switched away from this context but
+ * before the context has been removed from the execlist queue.
+ *
+ * 3. The driver has lost an interrupt. Typically the hardware has
+ * gone to idle but the driver still thinks the context belonging to
+ * the request at the head of the queue is still executing.
+ *
+ * CONTEXT_SUBMISSION_STATUS_NONE_SUBMITTED if no context has been found
+ * to be submitted to the execlist queue and if the hardware is idle.
+ */
+enum context_submission_status
+intel_execlists_TDR_get_current_request(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request **req)
+{
+ struct drm_i915_private *dev_priv;
+ unsigned long flags;
+ struct drm_i915_gem_request *tmpreq = NULL;
+ struct intel_context *tmpctx = NULL;
+ unsigned hw_context = 0;
+ unsigned sw_context = 0;
+ bool hw_active = false;
+ enum context_submission_status status =
+ CONTEXT_SUBMISSION_STATUS_UNDEFINED;
+
+ if (WARN_ON(!ring))
+ return status;
+
+ dev_priv = ring->dev->dev_private;
+
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+ spin_lock_irqsave(&ring->execlist_lock, flags);
+ hw_context = I915_READ(RING_EXECLIST_STATUS_CTX_ID(ring));
+
+ hw_active = (I915_READ(RING_EXECLIST_STATUS_LO(ring)) &
+ EXECLIST_STATUS_CURRENT_ACTIVE_ELEMENT_STATUS) ? true : false;
+
+ tmpreq = list_first_entry_or_null(&ring->execlist_queue,
+ struct drm_i915_gem_request, execlist_link);
+
+ if (tmpreq) {
+ sw_context = intel_execlists_ctx_id((tmpreq->ctx)->engine[ring->id].state);
+
+ /*
+ * Only acknowledge the request in the execlist queue if it's
+ * actually been submitted to hardware, otherwise there's the
+ * risk of a false inconsistency detection between the
+ * (unsubmitted) request and the idle hardware state.
+ */
+ if (tmpreq->elsp_submitted > 0) {
+ /*
+ * If the caller has not passed a non-NULL req
+ * parameter then it is not interested in getting a
+ * request reference back. Don't temporarily grab a
+ * reference since holding the execlist lock is enough
+ * to ensure that the execlist code will hold its
+ * reference all throughout this function. As long as
+ * that reference is kept there is no need for us to
+ * take yet another reference. The reason why this is
+ * of interest is because certain callers, such as the
+ * TDR hang checker, cannot grab struct_mutex before
+ * calling and because of that we cannot dereference
+ * any requests (DRM might assert if we do). Just rely
+ * on the execlist code to provide indirect protection.
+ */
+ if (req)
+ i915_gem_request_reference(tmpreq);
+
+ if (tmpreq->ctx)
+ tmpctx = tmpreq->ctx;
+ }
+ }
+
+ if (tmpctx) {
+ status = ((hw_context == sw_context) && hw_active) ?
+ CONTEXT_SUBMISSION_STATUS_OK :
+ CONTEXT_SUBMISSION_STATUS_INCONSISTENT;
+ } else {
+ /*
+ * If we don't have any queue entries and the
+ * EXECLIST_STATUS register points to zero we are
+ * clearly not processing any context right now
+ */
+ WARN((hw_context || hw_active), "hw_context=%x, hardware %s!\n",
+ hw_context, hw_active ? "not idle":"idle");
+
+ status = (hw_context || hw_active) ?
+ CONTEXT_SUBMISSION_STATUS_INCONSISTENT :
+ CONTEXT_SUBMISSION_STATUS_NONE_SUBMITTED;
+ }
+
+ if (req)
+ *req = tmpreq;
+
+ spin_unlock_irqrestore(&ring->execlist_lock, flags);
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ return status;
+}
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index de41ad6..d9acb31 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -29,7 +29,9 @@
/* Execlists regs */
#define RING_ELSP(ring) _MMIO((ring)->mmio_base + 0x230)
#define RING_EXECLIST_STATUS_LO(ring) _MMIO((ring)->mmio_base + 0x234)
+#define EXECLIST_STATUS_CURRENT_ACTIVE_ELEMENT_STATUS (0x3 << 14)
#define RING_EXECLIST_STATUS_HI(ring) _MMIO((ring)->mmio_base + 0x234 + 4)
+#define RING_EXECLIST_STATUS_CTX_ID(ring) RING_EXECLIST_STATUS_HI(ring)
#define RING_CONTEXT_CONTROL(ring) _MMIO((ring)->mmio_base + 0x244)
#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH (1 << 3)
#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT (1 << 0)
@@ -118,4 +120,16 @@ u32 intel_execlists_ctx_id(struct drm_i915_gem_object *ctx_obj);
void intel_lrc_irq_handler(struct intel_engine_cs *ring);
void intel_execlists_retire_requests(struct intel_engine_cs *ring);
+int intel_execlists_read_tail(struct intel_engine_cs *ring,
+ struct intel_context *ctx,
+ u32 *tail);
+
+int intel_execlists_write_head(struct intel_engine_cs *ring,
+ struct intel_context *ctx,
+ u32 head);
+
+int intel_execlists_read_head(struct intel_engine_cs *ring,
+ struct intel_context *ctx,
+ u32 *head);
+
#endif /* _INTEL_LRC_H_ */
diff --git a/drivers/gpu/drm/i915/intel_lrc_tdr.h b/drivers/gpu/drm/i915/intel_lrc_tdr.h
new file mode 100644
index 0000000..4520753
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_lrc_tdr.h
@@ -0,0 +1,36 @@
+/*
+ * Copyright © 2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef _INTEL_LRC_TDR_H_
+#define _INTEL_LRC_TDR_H_
+
+/* Privileged execlist API used exclusively by TDR */
+
+void intel_execlists_TDR_context_resubmission(struct intel_engine_cs *ring);
+
+enum context_submission_status
+intel_execlists_TDR_get_current_request(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request **req);
+
+#endif /* _INTEL_LRC_TDR_H_ */
+
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 4060acf..def0dcf 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -434,6 +434,88 @@ static void ring_write_tail(struct intel_engine_cs *ring,
I915_WRITE_TAIL(ring, value);
}
+int intel_ring_disable(struct intel_engine_cs *ring)
+{
+ WARN_ON(!ring);
+
+ if (ring && ring->disable)
+ return ring->disable(ring);
+ else {
+ DRM_ERROR("Ring disable not supported on %s\n", ring->name);
+ return -EINVAL;
+ }
+}
+
+int intel_ring_enable(struct intel_engine_cs *ring)
+{
+ WARN_ON(!ring);
+
+ if (ring && ring->enable)
+ return ring->enable(ring);
+ else {
+ DRM_ERROR("Ring enable not supported on %s\n", ring->name);
+ return -EINVAL;
+ }
+}
+
+int intel_ring_save(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req,
+ bool force_advance)
+{
+ WARN_ON(!ring);
+
+ if (ring && ring->save)
+ return ring->save(ring, req, force_advance);
+ else {
+ DRM_ERROR("Ring save not supported on %s\n", ring->name);
+ return -EINVAL;
+ }
+}
+
+int intel_ring_restore(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req)
+{
+ WARN_ON(!ring);
+
+ if (ring && ring->restore)
+ return ring->restore(ring, req);
+ else {
+ DRM_ERROR("Ring restore not supported on %s\n", ring->name);
+ return -EINVAL;
+ }
+}
+
+void intel_gpu_engine_reset_resample(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req)
+{
+ struct intel_ringbuffer *ringbuf;
+ struct drm_i915_private *dev_priv;
+
+ if (WARN_ON(!ring))
+ return;
+
+ dev_priv = ring->dev->dev_private;
+
+ if (i915.enable_execlists) {
+ struct intel_context *ctx;
+
+ if (WARN_ON(!req))
+ return;
+
+ ctx = req->ctx;
+ ringbuf = ctx->engine[ring->id].ringbuf;
+
+ /*
+ * In gen8+ context head is restored during reset and
+ * we can use it as a reference to set up the new
+ * driver state.
+ */
+ I915_READ_HEAD_CTX(ring, ctx, ringbuf->head);
+ ringbuf->last_retired_head = -1;
+ intel_ring_update_space(ringbuf);
+ }
+}
+
u64 intel_ring_get_active_head(struct intel_engine_cs *ring)
{
struct drm_i915_private *dev_priv = ring->dev->dev_private;
@@ -629,7 +711,7 @@ static int init_ring_common(struct intel_engine_cs *ring)
ringbuf->tail = I915_READ_TAIL(ring) & TAIL_ADDR;
intel_ring_update_space(ringbuf);
- memset(&ring->hangcheck, 0, sizeof(ring->hangcheck));
+ i915_hangcheck_reinit(ring);
out:
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 7349d92..7014778 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -49,6 +49,22 @@ struct intel_hw_status_page {
#define I915_READ_MODE(ring) I915_READ(RING_MI_MODE((ring)->mmio_base))
#define I915_WRITE_MODE(ring, val) I915_WRITE(RING_MI_MODE((ring)->mmio_base), val)
+
+#define I915_READ_TAIL_CTX(engine, ctx, outval) \
+ intel_execlists_read_tail((engine), \
+ (ctx), \
+ &(outval));
+
+#define I915_READ_HEAD_CTX(engine, ctx, outval) \
+ intel_execlists_read_head((engine), \
+ (ctx), \
+ &(outval));
+
+#define I915_WRITE_HEAD_CTX(engine, ctx, val) \
+ intel_execlists_write_head((engine), \
+ (ctx), \
+ (val));
+
/* seqno size is actually only a uint32, but since we plan to use MI_FLUSH_DW to
* do the writes, and that must have qw aligned offsets, simply pretend it's 8b.
*/
@@ -94,6 +110,34 @@ struct intel_ring_hangcheck {
enum intel_ring_hangcheck_action action;
int deadlock;
u32 instdone[I915_NUM_INSTDONE_REG];
+
+ /*
+ * Last recorded ring head index.
+ * This is only ever a ring index where as active
+ * head may be a graphics address in a ring buffer
+ */
+ u32 last_head;
+
+ /* Flag to indicate if engine reset required */
+ atomic_t flags;
+
+ /* Indicates request to reset this engine */
+#define I915_ENGINE_RESET_IN_PROGRESS (1<<0)
+
+ /*
+ * Timestamp (seconds) from when the last time
+ * this engine was reset.
+ */
+ u32 last_engine_reset_time;
+
+ /*
+ * Number of times this engine has been
+ * reset since boot
+ */
+ u32 reset_count;
+
+ /* Number of TDR hang detections */
+ u32 tdr_count;
};
struct intel_ringbuffer {
@@ -205,6 +249,14 @@ struct intel_engine_cs {
#define I915_DISPATCH_RS 0x4
void (*cleanup)(struct intel_engine_cs *ring);
+ int (*enable)(struct intel_engine_cs *ring);
+ int (*disable)(struct intel_engine_cs *ring);
+ int (*save)(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req,
+ bool force_advance);
+ int (*restore)(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req);
+
/* GEN8 signal/wait table - never trust comments!
* signal to signal to signal to signal to signal to
* RCS VCS BCS VECS VCS2
@@ -311,6 +363,9 @@ struct intel_engine_cs {
struct intel_ring_hangcheck hangcheck;
+ /* Saved head value to be restored after reset */
+ u32 saved_head;
+
struct {
struct drm_i915_gem_object *obj;
u32 gtt_offset;
@@ -463,6 +518,15 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
int intel_ring_space(struct intel_ringbuffer *ringbuf);
bool intel_ring_stopped(struct intel_engine_cs *ring);
+void intel_gpu_engine_reset_resample(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req);
+int intel_ring_disable(struct intel_engine_cs *ring);
+int intel_ring_enable(struct intel_engine_cs *ring);
+int intel_ring_save(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req, bool force_advance);
+int intel_ring_restore(struct intel_engine_cs *ring,
+ struct drm_i915_gem_request *req);
+
int __must_check intel_ring_idle(struct intel_engine_cs *ring);
void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index 2df4246..f20548c 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -1623,6 +1623,153 @@ bool intel_has_gpu_reset(struct drm_device *dev)
return intel_get_gpu_reset(dev) != NULL;
}
+static inline int wait_for_engine_reset(struct drm_i915_private *dev_priv,
+ unsigned int grdom)
+{
+#define _CND ((__raw_i915_read32(dev_priv, GEN6_GDRST) & grdom) == 0)
+
+ /*
+ * Spin waiting for the device to ack the reset request.
+ * Times out after 500 us
+ * */
+ return wait_for_atomic_us(_CND, 500);
+
+#undef _CND
+}
+
+static int do_engine_reset_nolock(struct intel_engine_cs *engine)
+{
+ int ret = -ENODEV;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+
+ assert_spin_locked(&dev_priv->uncore.lock);
+
+ switch (engine->id) {
+ case RCS:
+ __raw_i915_write32(dev_priv, GEN6_GDRST, GEN6_GRDOM_RENDER);
+ engine->hangcheck.reset_count++;
+ ret = wait_for_engine_reset(dev_priv, GEN6_GRDOM_RENDER);
+ break;
+
+ case BCS:
+ __raw_i915_write32(dev_priv, GEN6_GDRST, GEN6_GRDOM_BLT);
+ engine->hangcheck.reset_count++;
+ ret = wait_for_engine_reset(dev_priv, GEN6_GRDOM_BLT);
+ break;
+
+ case VCS:
+ __raw_i915_write32(dev_priv, GEN6_GDRST, GEN6_GRDOM_MEDIA);
+ engine->hangcheck.reset_count++;
+ ret = wait_for_engine_reset(dev_priv, GEN6_GRDOM_MEDIA);
+ break;
+
+ case VECS:
+ __raw_i915_write32(dev_priv, GEN6_GDRST, GEN6_GRDOM_VECS);
+ engine->hangcheck.reset_count++;
+ ret = wait_for_engine_reset(dev_priv, GEN6_GRDOM_VECS);
+ break;
+
+ case VCS2:
+ __raw_i915_write32(dev_priv, GEN6_GDRST, GEN8_GRDOM_MEDIA2);
+ engine->hangcheck.reset_count++;
+ ret = wait_for_engine_reset(dev_priv, GEN8_GRDOM_MEDIA2);
+ break;
+
+ default:
+ DRM_ERROR("Unexpected engine: %d\n", engine->id);
+ break;
+ }
+
+ return ret;
+}
+
+static int gen8_do_engine_reset(struct intel_engine_cs *engine)
+{
+ struct drm_device *dev = engine->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ int ret = -ENODEV;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+ ret = do_engine_reset_nolock(engine);
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+
+ if (!ret) {
+ u32 reset_ctl = 0;
+
+ /*
+ * Confirm that reset control register back to normal
+ * following the reset.
+ */
+ reset_ctl = I915_READ(RING_RESET_CTL(engine->mmio_base));
+ WARN(reset_ctl & 0x3, "Reset control still active after reset! (0x%08x)\n",
+ reset_ctl);
+ } else {
+ DRM_ERROR("Engine reset failed! (%d)\n", ret);
+ }
+
+ return ret;
+}
+
+int intel_gpu_engine_reset(struct intel_engine_cs *engine)
+{
+ /* Reset an individual engine */
+ int ret = -ENODEV;
+ struct drm_device *dev = engine->dev;
+
+ switch (INTEL_INFO(dev)->gen) {
+ case 8:
+ ret = gen8_do_engine_reset(engine);
+ break;
+ default:
+ DRM_ERROR("Per Engine Reset not supported on Gen%d\n",
+ INTEL_INFO(dev)->gen);
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * On gen8+ a reset request has to be issued via the reset control register
+ * before a GPU engine can be reset in order to stop the command streamer
+ * and idle the engine. This replaces the legacy way of stopping an engine
+ * by writing to the stop ring bit in the MI_MODE register.
+ */
+int intel_request_gpu_engine_reset(struct intel_engine_cs *engine)
+{
+ /* Request reset for an individual engine */
+ int ret = -ENODEV;
+ struct drm_device *dev = engine->dev;
+
+ if (INTEL_INFO(dev)->gen >= 8)
+ ret = gen8_request_engine_reset(engine);
+ else
+ DRM_ERROR("Reset request not supported on Gen%d\n",
+ INTEL_INFO(dev)->gen);
+
+ return ret;
+}
+
+/*
+ * It is possible to back off from a previously issued reset request by simply
+ * clearing the reset request bit in the reset control register.
+ */
+int intel_unrequest_gpu_engine_reset(struct intel_engine_cs *engine)
+{
+ /* Roll back reset request for an individual engine */
+ int ret = -ENODEV;
+ struct drm_device *dev = engine->dev;
+
+ if (INTEL_INFO(dev)->gen >= 8)
+ ret = gen8_unrequest_engine_reset(engine);
+ else
+ DRM_ERROR("Reset unrequest not supported on Gen%d\n",
+ INTEL_INFO(dev)->gen);
+
+ return ret;
+}
+
bool intel_uncore_unclaimed_mmio(struct drm_i915_private *dev_priv)
{
return check_for_unclaimed_mmio(dev_priv);
--
1.9.1
More information about the Intel-gfx
mailing list