Does gst-omx support Raspberry Pi for simultaneously H.264 video decoding and coding?

Thu Jan 28 13:55:45 PST 2016

Hi Miguel.

Sounds really good. Unfortunately I can't volunteer to do the coding for
the moment. Sorry for that. Hope somebody else can. One thing to remember
to get right, especially on a lower performance platform such as RaspPi,
would be for the module to request buffers for video from downstream and
use these for the cam to fill these, so we can avoid an extra memcpy(),
which is not insignificant for 1080p. Not sure how well that will fit with
mmal.

For the multipad, raw, encoded, stills and preview, then yes, the encoded
easily can be skipped and be replaced by omxh264dec, with no real
additional CPU work required, compared with current solution. Only reason
to include it should be, that the current module delivers h264, so to
maintain compatibility .... but the others, raw(RGB and YUV), stills and
preview may (do) have their own use.

Best regards
Peter

On Thu, Jan 28, 2016 at 10:30 PM, Miguel Domingues <
miguelbrazaodomingues at gmail.com> wrote:

> Hi,
>
> After looking at:
> https://github.com/thaytan/gst-rpicamsrc/blob/master/src/RaspiCapture.c
> from gst-rpicamsrc and raspivid.c and raspividyuv.c (both from rpi) it
> looks that it should be possible and relatively easy to adapt the
> gst-rpicamsrc to provide raw video.
>
> Getting simultaneously raw and h264 video, should also be possible
> although more difficult.
> However with the raw video you can always use the omxh264enc to encode it
> (e.g., gst-launch rpicamsrcraw ! ... ! omxh264enc ! ... ).
> Don't know if it would have a big impact on performance.
>
> Thanks Peter for the info regarding the encoding/decoding/encoding process.
> Nevertheless, just to resume the topic, it is possible to (at least)
> encode, decode and encode again simultaneously with the rpi.
>
> Regards,
> Miguel
>
>
> Miguel Domingues
>
> On Thu, Jan 28, 2016 at 11:40 AM, Peter Maersk-Moller <pmaersk at gmail.com>
> wrote:
>
>> Hi Miguel.
>>
>> In raspivid, they use mmal to pass video buffers between camera and
>> encoder.
>>
>> https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/RaspiVid.c
>>
>> In this example, they acess YUV components before passing to the encoder.
>> See video_record.c
>> https://github.com/tasanakorn/rpi-mmal-demo
>> I think you can get RGB as well.
>>
>> So it should be possible to both get raw video data and H.264 at the same
>> time. Would be cool if rpicamsrc used this to optinally ondemand output
>> multiple stream, raw, encoded, stills and preview.
>>
>> best regards
>> Peter Maersk-Moller
>>
>> On Thu, Jan 28, 2016 at 11:36 AM, Miguel Domingues <
>> miguelbrazaodomingues at gmail.com> wrote:
>>
>>> I was unaware that raspivid encoded the video.
>>>
>>> Regarding the pipeline, I split the HD 1080 video (with a tee), the
>>> first goes into a appsink to record the video when I want. The second copy
>>> is decoded, rescaled, changes fps and then is used for web streaming
>>> (re-encoded into h264) and motion detection (using opencv).
>>> The motion detection signals the recorder appsink to start and stop
>>> recording.
>>>
>>> So in the end, if rpicamsrc/raspivid is encoding the video, then my
>>> pipeline encodes, decodes and re-encodes...
>>> I will also look into raspivid to see if it is possible to get the raw
>>> video.
>>>
>>> Thanks for the info.
>>> Miguel
>>>
>>> Miguel Domingues
>>>
>>> On Thu, Jan 28, 2016 at 10:13 AM, Peter Maersk-Moller <pmaersk at gmail.com
>>> > wrote:
>>>
>>>> Hi Tim
>>>>
>>>> See inline.
>>>>
>>>> On Thu, Jan 28, 2016 at 10:55 AM, Tim Müller <tim at centricular.com>
>>>> wrote:
>>>>
>>>>> On Thu, 2016-01-28 at 01:57 +0100, Peter Maersk-Moller wrote:
>>>>>
>>>>> > The rpicamsrc is a wrapper around raspivid. The raspivid reads raw
>>>>> > video from the cam and encodes it for H.264 using the processors
>>>>> > hardware encoder. So in your pipeline you do two hardware encodings
>>>>> > and one decoding. Unless Broadcom did something really stupid, it
>>>>> > should be possible to modify rpicamsrc (using code example from
>>>>> > raspivid or raspiyuv) to read RGB or YUV (I420 perhaps).
>>>>>
>>>>> Are you sure it's possible to get the raw (post-processed) video from
>>>>> the camera sensors before it gets encoded? I seem to remember that the
>>>>> imaging pipeline is closed and they don't give you the option of
>>>>> getting RGB or YUV out, but maybe that's changed.
>>>>>
>>>>
>>>> That's why i wrote "Unless Broadcom did something really stupid ...".
>>>> Github was down last night so I couldn't check the source of raspivid -
>>>> and a little bit busy with a customer today.
>>>>
>>>> That said, raspiyuv can deliver stills as YUV or RGB. And after all,
>>>> video is a long row of successive stills so maybe there is a way.
>>>>
>>>> best regards
>>>> Peter
>>>>
>>>>
>>>>
>>>>>
>>>>>  Cheers
>>>>>   -Tim
>>>>>
>>>>> --
>>>>> Tim Müller, Centricular Ltd - http://www.centricular.com
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>
>>>>
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160128/c32ab475/attachment-0001.html>