Stream from linux framebuffer to h264 stream on Raspberry Pi

Fri Jul 12 00:58:28 PDT 2013

Hi,

I'd like to stream the framebuffer (/dev/fd0) on a Raspberry Pi to a h264
stream using the omxh264enc element.

I'm having some difficulty with this however.  The RPi's framebuffer in
Raspbian is using the RGB565 or possibly the BGR565 color space.

Here is the output of "fbset"

pi at raspberrypi ~ $ fbset -i

mode "1024x576"

    geometry 1024 576 1024 576 16

    timings 0 0 0 0 0 0 0

    accel true

    rgba 5/11,6/5,5/0,0/16

endmode

Frame buffer device information:

    Name        : BCM2708 FB

    Address     : 0x50006000

    Size        : 1179648

    Type        : PACKED PIXELS

    Visual      : TRUECOLOR

    XPanStep    : 0

    YPanStep    : 0

    YWrapStep   : 0

    LineLength  : 2048

    Accelerator : No

The closest I've gotten do doing this with just gstreamer is this pipeline:

pi at raspberrypi ~/videos/test $ gst-launch-1.0 -v --gst-debug 3 multifilesrc
location=/dev/fb0 num-buffers = 100 ! videoparse format=29 width=1024
height=576 framerate=25/1 ! decodebin ! videoconvert ! omxh264enc ! filesink
location=ss45.h264

0:00:00.438961309 12813   0xaa6440 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'

0:00:00.440577215 12813   0xaa6440 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

0:00:01.216074813 12813   0xa00a90 FIXME                default
gstutils.c:3622:gst_pad_create_stream_id_printf_valist:<multifilesrc0:src>
Creating random stream-id, consider implementing a deterministic way of
creating a stream-id

/GstPipeline:pipeline0/GstVideoParse:videoparse0.GstPad:src: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink.GstProxyPad:
proxypad0: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, width=(int)1024, height=(int)576, format=(string)I420,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:sink: caps = video/x-raw, width=(int)1024, height=(int)576,
format=(string)I420, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPa
d:proxypad1: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
profile=(string)high, level=(string)4, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
profile=(string)high, level=(string)4, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1 Pipeline is
PREROLLED ...

Setting pipeline to PLAYING ...

0:00:01.569037155 12813   0xae1860 ERROR            omxvideoenc
gstomxvideoenc.c:724:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh2
64enc0> No corresponding frame found

0:00:01.572056978 12813   0xae1860 ERROR            omxvideoenc
gstomxvideoenc.c:724:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh2
64enc0> No corresponding frame found

New clock: GstSystemClock

Got EOS from element "pipeline0".

Execution ended after 21189725790 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

Setting pipeline to NULL ...

Freeing pipeline ...

The above pipeline gives me a raw h264 video file.  I can't get a pipeline
that will work with any kind of useful muxer.  For instance, a mov:

pi at raspberrypi ~/videos/test $ gst-launch-1.0 --gst-debug 3 -v multifilesrc
location=/dev/fb0 num-buffers = 100 ! videoparse format=29 width=1024
height=576 framerate=25/1 ! decodebin ! videoconvert ! omxh264enc ! qtmux !
filesink location=ss46.h264.mov

0:00:00.478667830 16340  0x18dc440 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'

0:00:00.480339731 16340  0x18dc440 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

0:00:01.275239952 16340  0x1904400 FIXME                default
gstutils.c:3622:gst_pad_create_stream_id_printf_valist:<multifilesrc0:src>
Creating random stream-id, consider implementing a deterministic way of
creating a stream-id

/GstPipeline:pipeline0/GstVideoParse:videoparse0.GstPad:src: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink.GstProxyPad:
proxypad0: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, width=(int)1024, height=(int)576, framerate=(fraction)25/1,
format=(string)I420, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:sink: caps = video/x-raw, width=(int)1024, height=(int)576,
framerate=(fraction)25/1, format=(string)I420,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive

/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPa
d:proxypad1: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1

/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1

0:00:01.597787969 16340  0x1924e00 WARN                GST_PADS
gstpad.c:3577:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send
sticky events

/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1

0:00:01.611191180 16340  0x1924e00 WARN                GST_PADS
gstpad.c:3577:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send
sticky events

0:00:01.627075245 16340  0x1924e00 WARN             omxvideoenc
gstomxvideoenc.c:930:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error:
Internal data stream error.

0:00:01.628409167 16340  0x1924e00 WARN             omxvideoenc
gstomxvideoenc.c:930:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error:
stream stopped, reason not-negotiated

ERROR: from element
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
Internal data stream error.

Additional debug info:

gstomxvideoenc.c(930): gst_omx_video_enc_loop ():
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:

stream stopped, reason not-negotiated

ERROR: pipeline doesn't want to preroll.

/GstPipeline:pipeline0/GstQTMux:qtmux0.GstPad:src: caps = video/quicktime,
variant=(string)apple

/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/quicktime, variant=(string)apple Setting pipeline to NULL ...

Freeing pipeline ...

I have a feeling that videoparse isn't properly turning the raw video into
proper video frames.  I've also tried using ffmpeg's fbdev frame bufffer
source format to store the raw video wrapped as a mov.  

pi at raspberrypi ~/videos/test $ /usr/local/bin/ffmpeg -y -f fbdev -r 10 -i
/dev/fb0 -codec:v copy -frames 10 -f mov ss37.raw.mov

pi at raspberrypi ~/videos/test $ gst-launch-1.0 --gst-debug 3 -v filesrc
location=ss37.raw.mov ! decodebin ! videoconvert ! omxh264enc ! filesink
location=ss47.h264

0:00:00.396099711 22342  0x1caa300 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'

0:00:00.397728615 22342  0x1caa300 ERROR                    omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/quicktime

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/quicktime

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0.GstPad:si
nk: caps = video/quicktime

0:00:01.284890453 22342  0x1ce0c60 WARN                 qtdemux
qtdemux_types.c:192:qtdemux_type_get: unknown QuickTime node type L565

0:00:01.286235373 22342  0x1ce0c60 WARN                 qtdemux
qtdemux_types.c:192:qtdemux_type_get: unknown QuickTime node type -swr

0:00:01.298378659 22342  0x1ce0c60 WARN                 default
descriptions.c:998:gst_pb_utils_get_codec_description: No description
available for media type: video/x-gst-fourcc-L565

Missing element: video/x-gst-fourcc-L565 decoder

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-buffers = 5

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-time = 0

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-buffers = 5

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-time = 0

/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152

0:00:01.314092735 22342  0x1ce0c60 WARN               decodebin
gstdecodebin2.c:3703:gst_decode_bin_expose:<decodebin0> No suitable plugins
found

0:00:01.315684642 22342  0x1ce0c60 WARN               decodebin
gstdecodebin2.c:3705:gst_decode_bin_expose:<decodebin0> error: no suitable
plugins found

ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Your
GStreamer installation is missing a plug-in.

Additional debug info:

gstdecodebin2.c(3705): gst_decode_bin_expose ():
/GstPipeline:pipeline0/GstDecodeBin:decodebin0:

no suitable plugins found

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

Freeing pipeline ...

So it looks like decodebin cannot recognise video/x-gst-fourcc-L565 (which
is RGB16, I believe).

Any suggestions?

Dan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130712/dd8957e9/attachment-0001.html>