[Intel-gfx] [PATCH 3/8] drm/i915: Framework for capturing command stream based OA reports

[sourab gupta]

On Thu, 2017-03-16 at 02:03 -0700, Chris Wilson wrote:
> On Thu, Mar 16, 2017 at 02:24:55PM +0530, sourab gupta wrote:
> > On Thu, 2017-03-16 at 01:10 -0700, Chris Wilson wrote:
> > > On Thu, Mar 16, 2017 at 11:44:10AM +0530, sourab.gupta at intel.com wrote:
> > > > @@ -3593,6 +3670,7 @@ void i915_oa_init_reg_state(struct intel_engine_cs *engine,
> > > >  void i915_oa_update_reg_state(struct intel_engine_cs *engine,
> > > >  			      struct i915_gem_context *ctx,
> > > >  			      uint32_t *reg_state);
> > > > +void i915_perf_command_stream_hook(struct drm_i915_gem_request *req);
> > > >  
> > > >  /* i915_gem_evict.c */
> > > >  int __must_check i915_gem_evict_something(struct i915_address_space *vm,
> > > > diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > > > index aa75ea2..7af32c97 100644
> > > > --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > > > +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > > > @@ -1441,12 +1441,16 @@ static void eb_export_fence(struct drm_i915_gem_object *obj,
> > > >  	if (exec_len == 0)
> > > >  		exec_len = params->batch->size - params->args_batch_start_offset;
> > > >  
> > > > +	i915_perf_command_stream_hook(params->request);
> > > 
> > > Could you have named it anything more cyptic and inconsistent?
> > 
> > Sorry. Would 'i915_perf_capture_metrics' work?
> > Can you please suggest an appropriate name otherwise.
> 
> The verb we use for writting into the command stream is emit. So
> i915_perf_emit_samples() (emit_record record is clumsy as it is not clear
> whether it is the verb or noun).
> 
Thanks for suggesting. I'll use 'i915_perf_emit_samples' here.

> > > 
> > > Quite clearly this can fail, so justify the non handling of errors.
> > > 
> > > DRM_ERROR is not error handling, it is an indication that this patch
> > > isn't ready.
> > 
> > Well, the intent was to have minimal effect to execbuf normal routine,
> > even if we fail. But, I guess I'm mistaken.
> > I'll rectify this to handle the case, if perf callback fails.
> 
> That all depends on whether or not you can handle the unbalanced
> metrics. If simply missing a report is fine, then just kill the
> DRM_ERROR.
> 
> The bigger question is whether the following emit can to fail -- once
> the batch is in the request, no more failures are tolerable. You have to
> preallocate reserved space.
> 
> Don't you need a flush before the emit following the batch?
> 

Ok. So, that would mean that we have to first of all reserve the space
required by two 'perf_emit_samples' calls, so that we can't fail for the
lack of space in the emit following the batch.
Probably, I could pass an additional boolean parameter 'reserve_space'
set to true in the first call, which would reserve the space for both
emit_samples() calls (through intel_ring_begin)?


Would it still need the flush before the emit following the batch?
Ideally, the metrics should be collected as close to batch as possible.
So, if there are cache flush/tlb invalidate commands, it would introduce
some lag between the batch and following emit_samples command.
Sorry, I'm not able to gauge the need for flush here. I can understand
it's needed before the batch is programmed for flushing the cache/TLB
entries for the new workload to be submitted. But for the Sample_emit
commands, which generally only capture OA/timestamp/mmio metrics, is
this still required? 


> > > > +void i915_perf_command_stream_hook(struct drm_i915_gem_request *request)
> > > > +{
> > > > +	struct intel_engine_cs *engine = request->engine;
> > > > +	struct drm_i915_private *dev_priv = engine->i915;
> > > > +	struct i915_perf_stream *stream;
> > > > +
> > > > +	if (!dev_priv->perf.initialized)
> > > > +		return;
> > > > +
> > > > +	mutex_lock(&dev_priv->perf.streams_lock);
> > > 
> > > Justify a new global lock very, very carefully on execbuf.
> > 
> > The lock introduced here is to protect the the perf.streams list against
> > addition/deletion while we're processing the list during execbuf call.
> > The other places where the mutex is taken is when a new stream is being
> > created (using perf_open ioctl) or a stream is being destroyed
> > (perf_release ioctl), which just protect the list_add and list_del into
> > the perf.streams list.
> > As such, there should not be much impact on execbuf path.
> > Does this seem reasonable?
> 
> It doesn't sound like it needs a mutex, which will simplify the other
> users as well (if switched to, for example, an RCU protected list).

Ok. Sounds reasonable, I'll switch to using an RCU protected list here.

> 
> > > > +	list_for_each_entry(stream, &dev_priv->perf.streams, link) {
> > > > +		if ((stream->state == I915_PERF_STREAM_ENABLED) &&
> > > > +					stream->cs_mode)
> > > > +			stream->ops->command_stream_hook(stream, request);
> > > > +	}
> > > > +	mutex_unlock(&dev_priv->perf.streams_lock);
> > > > +}
> > > 
> > > > +static void i915_perf_command_stream_hook_oa(struct i915_perf_stream *stream,
> > > > +					struct drm_i915_gem_request *request)
> > > > +{
> > > > +	struct drm_i915_private *dev_priv = request->i915;
> > > > +	struct i915_gem_context *ctx = request->ctx;
> > > > +	struct i915_perf_cs_sample *sample;
> > > > +	u32 addr = 0;
> > > > +	u32 cmd, *cs;
> > > > +
> > > > +	sample = kzalloc(sizeof(*sample), GFP_KERNEL);
> > > > +	if (sample == NULL) {
> > > > +		DRM_ERROR("Perf sample alloc failed\n");
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	cs = intel_ring_begin(request, 4);
> > > > +	if (IS_ERR(cs)) {
> > > > +		kfree(sample);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	sample->ctx_id = ctx->hw_id;
> > > > +	i915_gem_request_assign(&sample->request, request);
> > > 
> > > > +
> > > > +	i915_gem_active_set(&stream->last_request, request);
> > > 
> > > Hint, you don't need you own request trap.
> > Sorry, I didn't understand you fully here. I'm storing the reference to
> > the last active request associated with the stream inside the
> > last_request 'gem_active' field. Do I need to do something differently
> > here?
> 
> It's the duplication.

Are you suggesting that request_assign() and active_set() is
duplication? 
Actually, I'm using the last_request active tracker for the purpose of
waiting on last request to complete, whenever the need.
But still I need to get reference for every request for which the
commands for collection of metrics are emitted. This is because I need
to check for their completion before collecting the associated metrics.
This is done inside 'append_command_stream_samples' function, which also
takes care of releasing the reference taken here.
Am I missing something w.r.t. the active_set() functionality?

> -Chris
> 

Thanks,
Sourab