BGRA for omxh264enc and omxh264dec

[Peter Maersk-Moller]

Hi Sebastian.

That is exactly what is needed. So it can be done in HW.

Any remote chance this gets into a release in future not too distant future
before practical realization of teleportation, hot fusion and singularity?

If not, what is the latest GStreamer version this works with and are there
some instructions somehwree on how merge this with some version on gst-omx ?

I am preparing a release of Snowmix that will make simple and advanced 2D
and 3D scriptable video mixing possible and practical on the Raspberry Pi 2
platform and I have all the other components necessary working pretty well
and efficiently except it just can't fly unless I can find a way to convert
a video stream to a RGBA frame stream without a crippling load on the CPU.
The *omxcolorscale*, either as an independent module or buit-in within the
omxXXXXdec module, is all that is needed. Same for omxXXXXenc would be nice
too, but converting from RGBA to I420 in software does not have same
perfomance penalty as the other way around with current implementation for
Raspberry Pi.

The release of Raspberry PI 3 with 30-100% more juice depending on what is
measured, just makes it all more realistic to do serious video mixing on
these low cost platforms.

I'm trying to read through the code, but would have no idea on how to
add/merge it to/with current gst-omx sources. Not that I can't add code,
just that I don't know all the internals og Gstreamer.

Best regards
Peter Maersk-Moller

On Mon, Mar 14, 2016 at 9:06 AM, Sebastian Dröge <sebastian at centricular.com>
wrote:

> On Mo, 2016-03-14 at 05:39 +0100, Peter Maersk-Moller wrote:
> > Hi Sebastian.
> >
> > Looking into the to the problem of converting and/or resize I420 or
> > NV12 output from the omxh264dec (and others), there seem to be ways
> > to both convert to other raw formats as well as perhaps resize the
> > output all using the GPU.
> >
> > The OpenMAX Il-component video_splitter (input port 250, output ports
> > 251-254 - only one is needed) can take input as I420 and output as
> > OMX_COLOR_Format32bitARGB8888 or OMX_COLOR_Format32bitBGRA8888 (one
> > of which I believe is called a byte stream of R, G, B and A bytes
> > (RGBA)) and possibly OMX_COLOR_Format24bitRGB888 or
> > OMX_COLOR_Format24bitBGR888 (one of which is believed to be RGB). I'm
> > not sure it can do both RGB(A) and BGR(A).
> >
> > Anyway, it should be possible to connect the encoder's port 131 to
> > the video_splitter's port 250, set the splitter port 250 format to
> > I420, perhaps provide buffers for this, set the splitter port 251 to
> > RGBA or RGB and feed it data buffers from downstream (perhaps shm)
> > for zero-copying. An of course the video_splitter component has to be
> > started and prepared for state chage etc.
> >
> > For video resizing (and conversion) the OpenMAX image component
> > resize can take I420 input on port 60 AND scale AND convert to image
> > formats such as RGBA and RGB (I am 95% sure) and output these to
> > output port 61 on data buffers from downstreams (perhaps shm).
> >
> > All this of course applies to the other omxXXXdec modules.
> >
> > The same should be possible for the input for omxh264enc and the
> > other encoders in a way so the encoders can take I420/RGB/RGBA at any
> > size and the decoders can output I420/RGB/RGBA at any size.
> >
> > The "any size" may be to take it to far. There are some restrictions
> > for either the encoder and the decoder. For one of them, the width
> > and height must both be even numbers and for the other, both must be
> > a multipla of 16. I just don't know if you can link OpenMAX video
> > modules to OpenMAX image modules. At least some timing may be set and
> > controlled out-of-band when going over an image module .... perhaps.
> >
> > I tried looking to the GStreamer code, but I can't really see where
> > to add the functionality and I am still struggling with the very
> > poorly documented OpenMAX examples and docs.
> >
> > I hope this is worth looking at. Especailly converting from I420 to
> > RGBA for is nearly impossible on the Pi-platform for 720p and quite
> > impossible for 1080p with the current gstreamer version, conversion
> > alone take a lot of CPU not really present.
> >
> > Does it help to open a ticket for an improvement?
>
> Here's a ticket with an (outdated) patch that integrates the resize
> component via tunneling into the decoder:
> https://bugzilla.gnome.org/show_bug.cgi?id=722686
>
> --
> Sebastian Dröge, Centricular Ltd · http://www.centricular.com
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160314/919e90f0/attachment.html>