Proposal for a low-level Linux display framework

[Rob Clark]

On Sun, Sep 18, 2011 at 5:23 PM, Alan Cox <alan at lxorguk.ukuu.org.uk> wrote:
>> This would leave us with the issue of controlling formats and other parameters
>> on the pipelines. We could keep separate DRM, KMS, FB and V4L APIs for that,
>
> There are some other differences that matter. The exact state and
> behaviour of memory, sequencing of accesses, cache control and management
> are a critical part of DRM for most GPUs, as is the ability to have them
> in swap backed objects and to do memory management of them. Fences and
> the like are a big part of the logic of many renderers and the same
> fencing has to be applied between capture and GPU, and also in some cases
> between playback accelerators (eg MP4 playback) and GPU.
>
> To glue them together I think you'd need to support the use of GEM objects
> (maybe extended) in V4L. That may actually make life cleaner and simpler
> in some respects because GEM objects are refcounted nicely and have
> handles.

fwiw, I think the dmabuf proposal that linaro GWG is working on should
be sufficient for V4L to capture directly into a GEM buffer that can
be scanned out (overlay) or composited by GPU, etc, in cases where the
different dma initiators can all access some common memory:

http://lists.linaro.org/pipermail/linaro-mm-sig/2011-September/000616.html

The idea is that you could allocate a GEM buffer, export a dmabuf
handle for that buffer that could be passed to v4l2 camera device (ie.
V4L2_MEMORY_DMABUF), video encoder, etc..  the importing device should
bracket DMA to/from the buffer w/ get/put_scatterlist() so an unused
buffer could be unpinned if needed.

> DRM and KMS abstract out stuff into what is akin to V4L subdevices for
> the various objects the video card has that matter for display - from
> scanout buffers to the various video outputs, timings and the like.
>
> I don't know what it's like with OMAP but for some of the x86 stuff
> particularly low speed/power stuff the capture devices, GPU and overlays
> tend to be fairly incestuous in order to do things like 1080i/p preview
> while recording from the camera.

We don't like extra memcpy's, but something like dmabuf fits us
nicely.. and I expect it would work well in any sort of UMA system
where camera, encoder, GPU, overlay, etc all can share the same memory
and formats.  I suspect the situation is similar in the x86 SoC
world.. but would be good to get some feedback on the proposal.  (I
guess next version of the RFC would go out to more mailing lists for
broader review.)

BR,
-R

> GPU is also a bit weird in some ways because while its normally
> nonsensical to do things like use the capture facility one card to drive
> part of another, it's actually rather useful (although not supported
> really by DRM) to do exactly that with GPUs. A simple example is a dual
> headed box with a dumb frame buffer and an accelerated output both of
> which are using memory that can be hit by the accelerated card. Classic
> example being a USB plug in monitor.
>
> _______________________________________________
> linaro-dev mailing list
> linaro-dev at lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/linaro-dev
>