[gst-devel] Using cairo/pixman for raw video in GStreamer

[David Schleef]

On Fri, Sep 11, 2009 at 07:02:48PM +0200, Benjamin Otte wrote:
> Hi,
> 
> This is an idea that's been brewing in my head for a bit. After
> thinking about it for a while and poking some people on IRC, I'm
> pretty convinced it's the best way forward.
> 
> Here's a list of problems I'd like to see solved:
> 
> 1) Correctly identify video in the GStreamer elements (stride, width,
> height, size of image and components)
> In the short while I recently hacked on plugins, I found bugs in lots
> of places, from common to obscure formats. And those were in pretty
> common elements (theoraenc/dec, videotestsrc). Using the APIs in
> gstvideo pretty much solves this problem for the current set of
> plugins.

Just because an element is common doesn't mean it's been reviewed
in several years.  videotestsrc predates gstvideo, and hasn't been
updated to use gstvideo.  It should.  But I've been waiting until
I've moved some additional video frame stuff from Cog/Schroedinger
into GStreamer.  The cog/schro is well-tested and calculation of
frame component sizes is more obvious than in videotestsrc.  Also,
it supports more formats.

> 2) Allow drawing onto different video formats
> There is actually multiple issues here: For a start, elements that
> draw to various YUV formats often get it wrong - mostly in corner
> cases. Others take shortcuts that degrade the quality of the video
> (like videotestsrc not computing the average for U and V pixels for
> subsampled planes).

See cog.

> 4) Allow hw-acceleration in the video pipeline
> Decoding a H264 stream in hardware, rendering sutitles on top of it,
> scaling it to fit and displaying as fullscreen video on my computer
> can in theory all be done in hardware. Unfortunately, GStreamer
> currently lacks infrastructure for this, so all this stuff ends up
> being done in software.

What can be implemented has been implemented in gst-plugins-gl, and
works relatively well.  It needs to be connected with VDPAU/VAAPI,
but that's something that still requires work at a lower level.



dave...