[Intel-gfx] [PATCH 2/2] drm/i915/tracepoints: Remove DRM_I915_LOW_LEVEL_TRACEPOINTS Kconfig option

Wed Aug 29 14:51:31 UTC 2018

Quoting Kukanova, Svetlana (2018-08-27 16:37:14)
> > Once there is an actual request to have some metrics from vanilla kernels
> > through some end-user tools (not a developer tool, like here), I'll be glad
> > to discuss about how to provide the information the best for them in a
> > stable manner.
> 
> Sorry for my ignorance, but looks like I don't understand what developer vs.
> end-user means here.
> With regard to GPU profiling VTune's end-user is somebody who develops gfx or
> media applications basing on MediaSDK, OpenCL, C for Media, etc.
> Or, more often it's an intel application engineer working with those people's
> code.
> AE in his\her turn may contact e.g. Dmitry's team if judging by VTune data
> he\she decides that the problem is on the deeper level of the gfx stack, not
> in the customer's code.
> Then Dmitry's team would be experimenting with VTune and deciding if the
> problem is in their code or it's deeper in i915.
> Don't think that i915 people use VTune (sadly:)) so here the chain is broken.
> Otherwise they could e.g. blame HW based on the same data.
> I'm wondering who in this chain (app developer, AE, Dmitry, i915) is an
> "end-user" and who's a "developer"?
> Or is a "developer" a kernel developer only?
> And e.g. Dmitry is an end-user and thus he is not supposed to use tools like
> gpuvis or VTune?
> Looks like all the chain before i915 is annoyed by the kernel-rebuilding
> requirement.

With end-user tool I'm referring to something that would have interest
in being packaged and shipped by a distro.

gpuvis team seems to be doing fine with the application being built from source
and being run against a specially configured kernel for their purposes. I would
assume there to be some queries about a enabling the tracepoints by default if
there was demand. At the same time I would assume them to try to get the
application packaged and into distros.

And then we would commence discussing how to provide the information in a stable
manner (most likely outside tracepoints). So far I'm not seeing such queries
from gpuvis direction.

> > The interface discussion would probably start from a DRM subsystem level, so
> > that the tool would have an equivalent level of base experience from all
> > drivers.
>
> That sounds like a solution from an ideal world. I mean if DRM had a uAPI for
> scheduling observability and all the drivers had to implement this. And the
> drivers would require info from HW like GuC pointing to the necessity of uAPI
> support...
> Would be just great for all the tools (, developers and end-users).
> But I have no idea what kind of impulse should it be to bring this to reality.
> And if all the energy available to human kind at the given evolution point
> would be enough to at least start this. 
> Or am I just too pessimistic? Are there some simple defined steps to be done
> to make it? Can we build a realistic plan?

Step is "1. Have the tool" :) There seem to be three options: 1) open sourcing
VTune 2) contributing to gpuvis project to drive the project into the above
mentioned direction. 3) writing a new project from scratch (not encouraged,
unless you have something differentiating to bring to the table).

Unless somebody actively drives the feature to some Open Source userspace
consumer, there won't be an interface for the information from kernel.
Demand from an Open Source application is a hard requirement for kickstarting
the interface discussion.

> E.g. is this the first step? -
> > There's just no Open Source tool to first design and then validate the
> > interfaces against. There's just the debugging tool which happens to work
> > currently, without any guarantees that next kernel version would not cause a
> > substantial rework of the interfacing code.
> 
> How does it usually work, I mean you can't have a widely shipped open-source
> consumer already using a non-existent feature that is to be requested? 
> And I can't imagine what kind of existing tool should it be to decide suddenly
> that it needs to add GPU scheduling tracing to the list of its features.
> If you want to have a new tool for GPU scheduling timeline - and it sounds
> like a sane idea, looks like we agree on the use cases etc. - how can you make
> it open source first and then get the API to be based on from i915?

The order is that you develop the tool and the required kernel changes
in parallel in topic branches, to demonstrate the usefulness of the tool
and suitability of the kernel interface. Then after all the patches are
reviewed (kernel + tool), kernel side is merged first, and then the tool
can start working from next kernel release.

This has been attempted to be described in the following
documentation chapter:

https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-uapi.html#open-source-userspace-requirements

> Or am I just missing the point completely?
> If the open-sourced MediaSDK was shipped with some distro (isn't it, btw?) -
> would Dmitry be eligible to request observability features for tools?

MediaSDK should not have anything to do with this, unless it will directly
consume the kernel interface in discussion.

The need is for some application/library/whatever userspace component to
demonstrate the suitability of the kernel interface and act as a counterpart
for the kernel interface that can be tested and debugged for changes.
This too, is explained in more detail in the above linked documentation
chapter.

Regards, Joonas

> 
> Thank you,
> Svetlana
> 
> -----Original Message-----
> From: Joonas Lahtinen [mailto:joonas.lahtinen at linux.intel.com] 
> Sent: Tuesday, August 21, 2018 3:07 PM
> To: Intel-gfx at lists.freedesktop.org; Kukanova, Svetlana <svetlana.kukanova at intel.com>; Chris Wilson <chris at chris-wilson.co.uk>; Tvrtko Ursulin <tursulin at ursulin.net>; Tvrtko Ursulin <tvrtko.ursulin at linux.intel.com>
> Subject: RE: [Intel-gfx] [PATCH 2/2] drm/i915/tracepoints: Remove DRM_I915_LOW_LEVEL_TRACEPOINTS Kconfig option
> 
> Quoting Kukanova, Svetlana (2018-08-13 16:44:49)
> > Joonas, sorry for interfering; could you please explain more regarding 
> > the options for tracing scheduling events better than tracepoints?
> > After scheduling moves to GuC tools will have to switch to something 
> > like GuC-logging; but while kmd does scheduling isn't kernel-tracing the best solution?
> > I know gpuvis is not the only attempt to use tracepoints for the same purpose.
> > (there're trace.pl and S.E.A. and of course VTune though it probably 
> > is not considered to be existing as it's not open source).
> > And assuming this movement towards GuC is it not too late to invent a 
> > completely new way to provide tools with scheduling info from kmd?
> > Could we just improve the existing way and let it live its last years\months? 
> 
> Hi,
> 
> You actually mentioned the prime reason why we should not go and hastily make tracepoints a stable uAPI with regards to scheduling information.
> 
> The scheduler's nature will be evolving when some of the scheduling decisions are moved to GuC and the way how we get the information will be changing at that point, so tracepoints will indeed be a very bad mechanism for providing the information.
> 
> The kernel scheduler is definitely not going anywhere with the introduction of more hardware scheduling capabilities, so it is a misconception to think that the interface would need to be completely different for when GuC is enabled.
> 
> > 
> > gpuvis works w\o modifying kernel for AMDgpu showing HW queue and HW 
> > execution; it cosplays Microsoft GPUView which works out-of-the-box on Windows too.
> > Thus it appears that intel gfx on linux is the most closed platform, 
> > not bothering of observability (or even bothering about how to forbid observability).
> 
> gpuvis is a developer tool. The tracepoints behind this configure switch are way more low-level than what the gpuvis seems to support for AMDGPU *at all*. They seem to stick to IOCTL level. So from what I see, we should be on-par with the competition even without any special kernel configuration. So lets not get things mixed up.
> 
> And I remind, the tool is not shipping anywhere really (except the AUR), but just built from source by developers in need, and they seem to be just fine with re-compiling the kernel (as there have been no requests).
> 
> Once there is an actual request to have some metrics from vanilla kernels through some end-user tools (not a developer tool, like here), I'll be glad to discuss about how to provide the information the best for them in a stable manner.
> 
> > Not long ago the MediaSDK team diagnosed a problem with their 
> > workloads looking at VTune timelines - seeing the difference between 
> > the time request came to kmd and time it went runnable & comparing the 
> > queues on 2 engines they understood that their requests have 
> > dependencies that were definitely unexpected. MediaSDK reported the problem to driver people and it was fixed.
> > 
> > I can add Dmitry Rogozhkin to discussion if the usefulness of 
> > scheduling timeline in tools is questionable, as far as I remember 
> > this wasn't the only use case they had, I'm sure he can add more.
> 
> I'm well aware of the use cases. And Dmitry is well aware of the need for an Open Source consumer for any requested stable uAPIs. And we don't currently have that, so there's no disconnect on information.
> 
> There's just no Open Source tool to first design and then validate the interfaces against. There's just the debugging tool which happens to work currently, without any guarantees that next kernel version would not cause a substantial rework of the interfacing code.
> 
> The interface discussion would probably start from a DRM subsystem level, so that the tool would have an equivalent level of base experience from all drivers.
> 
> Regards, Joonas
> 
> > 
> > Thank you,
> > Svetlana
> > 
> > -----Original Message-----
> > From: Intel-gfx [mailto:intel-gfx-bounces at lists.freedesktop.org] On 
> > Behalf Of Joonas Lahtinen
> > Sent: Monday, August 13, 2018 12:55 PM
> > To: Chris Wilson <chris at chris-wilson.co.uk>; 
> > Intel-gfx at lists.freedesktop.org; Tvrtko Ursulin 
> > <tursulin at ursulin.net>; Tvrtko Ursulin 
> > <tvrtko.ursulin at linux.intel.com>
> > Subject: Re: [Intel-gfx] [PATCH 2/2] drm/i915/tracepoints: Remove 
> > DRM_I915_LOW_LEVEL_TRACEPOINTS Kconfig option
> > 
> > Quoting Tvrtko Ursulin (2018-08-08 15:56:01)
> > > On 08/08/2018 13:42, Chris Wilson wrote:
> > > > Quoting Tvrtko Ursulin (2018-08-08 13:13:08)
> > > This is true, no disagreement. My point simply was that we can 
> > > provide this info easily to anyone. There is a little bit of analogy 
> > > with perf scheduler tracing/map etc.
> > > 
> > > > I don't see any value in giving the information away, just the cost. 
> > > > If you can convince Joonas of its merit, and if we can define just 
> > > > exactly what ABI it constitutes, then I'd be happy to be the one 
> > > > who says "I told you so" in the future for a change.
> > > 
> > > I think Joonas was okay in principle that we soft-commit to _trying_ 
> > > to keep _some_ tracepoint stable-ish (where it makes sense and after 
> > > some discussion for each) if IGT also materializes which auto-pings 
> > > us (via
> > > CI) when we break one of them. But I may be misremembering so Joonas 
> > > please comment.
> > 
> > Currently gpuvis, using these, seems to be only packaged in one AUR repo, and they do make a not in the wiki how you need to configure kernel for debugging. And there's been no apparent demand for them to have it in stock kernel.
> > 
> > And even when we do get demand for having gpuvis or another tool working from vanilla kernel, tracepoints being a rather tricky subject, I would start the discussion by going through alternative means of providing the information the tool needs and considering those.
> > 
> > So lets still keep this option as it was introduced. The whole "tracepoints as stable uAPI" idea is a can of worms which is only dug into when other options are exhausted.
> > 
> > Regards, Joonas
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx