Summary of the V4L2 discussions during LDS - was: Re: Embedded Linux memory management interest group list

[Mauro Carvalho Chehab]

Em 18-04-2011 17:15, Jesse Barker escreveu:
> One of the big issues we've been faced with at Linaro is around GPU
> and multimedia device integration, in particular the memory management
> requirements for supporting them on ARM.  This next cycle, we'll be
> focusing on driving consensus around a unified memory management
> solution for embedded systems that support multiple architectures and
> SoCs.  This is listed as part of our working set of requirements for
> the next six-month cycle (in spite of the URL, this is not being
> treated as a graphics-specific topic - we also have participation from
> multimedia and kernel working group folks):
> 
>   https://wiki.linaro.org/Cycles/1111/TechnicalTopics/Graphics

As part of the memory management needs, Linaro organized several discussions
during Linaro Development Summit (LDS), at Budapest, and invited me and other
members of the V4L and DRI community to discuss about the requirements.
I wish to thank Linaro for its initiative.

Basically, on several SoC designs, the GPU and the CPU are integrated into
the same chipset and they can share the same memory for a framebuffer. Also,
they may have some IP blocks that allow processing the framebuffer internally,
to do things like enhancing the image and converting it into an mpeg stream.

The desire, from the SoC developers, is that those operations should be
done using zero-copy transfers.

This resembles somewhat the idea of the VIDIOC_OVERLAY/VIDIOC_FBUF API, 
that was used in the old days where CPUs weren't fast enough to process
video without generating a huge load on it. So the overlay mode were created
to allow direct PCI2PCI transfers from the video capture board into the
display adapter, using XVideo extension, and removing the overload at the
CPU due to a video stream. It were designed as a Kernel API for it, and an
userspace X11 driver, that passes a framebuffer reference to the V4L driver,
where it is used to program the DMA transfers to happen inside the framebuffer.

At the LDS, we had a 3-day discussions about how the buffer sharing should
be handled, and Linaro is producing a blueprint plan to address the needs.
We had also a discussion about V4L and KMS, allowing both communities to better
understand how things are supposed to work on the other side.