Stream from linux framebuffer to h264 stream on Raspberry Pi
Dan Flett
dflett at bigpond.com
Thu Jul 11 23:55:46 PDT 2013
Hi,
I'd like to stream the framebuffer (/dev/fd0) on a Raspberry Pi to a h264
stream using the omxh264enc element.
I'm having some difficulty with this however. The RPi's framebuffer in
Raspbian is using the RGB565 or possibly the BGR565 color space.
Here is the output of "fbset"
pi at raspberrypi ~ $ fbset -i
mode "1024x576"
geometry 1024 576 1024 576 16
timings 0 0 0 0 0 0 0
accel true
rgba 5/11,6/5,5/0,0/16
endmode
Frame buffer device information:
Name : BCM2708 FB
Address : 0x50006000
Size : 1179648
Type : PACKED PIXELS
Visual : TRUECOLOR
XPanStep : 0
YPanStep : 0
YWrapStep : 0
LineLength : 2048
Accelerator : No
The closest I've gotten do doing this with just gstreamer is this pipeline:
pi at raspberrypi ~/videos/test $ gst-launch-1.0 -v --gst-debug 3 multifilesrc
location=/dev/fb0 num-buffers = 100 ! videoparse format=29 width=1024
height=576 framerate=25/1 ! decodebin ! videoconvert ! omxh264enc ! filesink
location=ss45.h264
0:00:00.438961309 12813 0xaa6440 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'
0:00:00.440577215 12813 0xaa6440 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
0:00:01.216074813 12813 0xa00a90 FIXME default
gstutils.c:3622:gst_pad_create_stream_id_printf_valist:<multifilesrc0:src>
Creating random stream-id, consider implementing a deterministic way of
creating a stream-id
/GstPipeline:pipeline0/GstVideoParse:videoparse0.GstPad:src: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink.GstProxyPad:
proxypad0: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, width=(int)1024, height=(int)576, format=(string)I420,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:sink: caps = video/x-raw, width=(int)1024, height=(int)576,
format=(string)I420, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPa
d:proxypad1: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
profile=(string)high, level=(string)4, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
profile=(string)high, level=(string)4, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:01.569037155 12813 0xae1860 ERROR omxvideoenc
gstomxvideoenc.c:724:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh2
64enc0> No corresponding frame found
0:00:01.572056978 12813 0xae1860 ERROR omxvideoenc
gstomxvideoenc.c:724:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh2
64enc0> No corresponding frame found
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 21189725790 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
The above pipeline gives me a raw h264 video file. I can't get a pipeline
that will work with any kind of useful muxer. For instance, a mov:
pi at raspberrypi ~/videos/test $ gst-launch-1.0 --gst-debug 3 -v multifilesrc
location=/dev/fb0 num-buffers = 100 ! videoparse format=29 width=1024
height=576 framerate=25/1 ! decodebin ! videoconvert ! omxh264enc ! qtmux !
filesink location=ss46.h264.mov
0:00:00.478667830 16340 0x18dc440 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'
0:00:00.480339731 16340 0x18dc440 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
0:00:01.275239952 16340 0x1904400 FIXME default
gstutils.c:3622:gst_pad_create_stream_id_printf_valist:<multifilesrc0:src>
Creating random stream-id, consider implementing a deterministic way of
creating a stream-id
/GstPipeline:pipeline0/GstVideoParse:videoparse0.GstPad:src: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink.GstProxyPad:
proxypad0: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, width=(int)1024, height=(int)576, framerate=(fraction)25/1,
format=(string)I420, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:sink: caps = video/x-raw, width=(int)1024, height=(int)576,
framerate=(fraction)25/1, format=(string)I420,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPa
d:proxypad1: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1024,
height=(int)576, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0,
framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps =
video/x-raw, format=(string)RGB16, width=(int)1024, height=(int)576,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)1:1:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1
0:00:01.597787969 16340 0x1924e00 WARN GST_PADS
gstpad.c:3577:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send
sticky events
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPa
d:src: caps = video/x-h264, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string)4,
width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1
0:00:01.611191180 16340 0x1924e00 WARN GST_PADS
gstpad.c:3577:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send
sticky events
0:00:01.627075245 16340 0x1924e00 WARN omxvideoenc
gstomxvideoenc.c:930:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error:
Internal data stream error.
0:00:01.628409167 16340 0x1924e00 WARN omxvideoenc
gstomxvideoenc.c:930:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> error:
stream stopped, reason not-negotiated
ERROR: from element
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
Internal data stream error.
Additional debug info:
gstomxvideoenc.c(930): gst_omx_video_enc_loop ():
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
stream stopped, reason not-negotiated
ERROR: pipeline doesn't want to preroll.
/GstPipeline:pipeline0/GstQTMux:qtmux0.GstPad:src: caps = video/quicktime,
variant=(string)apple
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/quicktime, variant=(string)apple
Setting pipeline to NULL ...
Freeing pipeline ...
I have a feeling that videoparse isn't properly turning the raw video into
proper video frames. I've also tried using ffmpeg's fbdev frame bufffer
source format to store the raw video wrapped as a mov.
pi at raspberrypi ~/videos/test $ /usr/local/bin/ffmpeg -y -f fbdev -r 10 -i
/dev/fb0 -codec:v copy -frames 10 -f mov ss37.raw.mov
pi at raspberrypi ~/videos/test $ gst-launch-1.0 --gst-debug 3 -v filesrc
location=ss37.raw.mov ! decodebin ! videoconvert ! omxh264enc ! filesink
location=ss47.h264
0:00:00.396099711 22342 0x1caa300 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element
'omxtheoradec'
0:00:00.397728615 22342 0x1caa300 ERROR omx
gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element
'omxvp8dec'
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/quicktime
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.G
stPad:src: caps = video/quicktime
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0.GstPad:si
nk: caps = video/quicktime
0:00:01.284890453 22342 0x1ce0c60 WARN qtdemux
qtdemux_types.c:192:qtdemux_type_get: unknown QuickTime node type L565
0:00:01.286235373 22342 0x1ce0c60 WARN qtdemux
qtdemux_types.c:192:qtdemux_type_get: unknown QuickTime node type -swr
0:00:01.298378659 22342 0x1ce0c60 WARN default
descriptions.c:998:gst_pb_utils_get_codec_description: No description
available for media type: video/x-gst-fourcc-L565
Missing element: video/x-gst-fourcc-L565 decoder
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
0:00:01.314092735 22342 0x1ce0c60 WARN decodebin
gstdecodebin2.c:3703:gst_decode_bin_expose:<decodebin0> No suitable plugins
found
0:00:01.315684642 22342 0x1ce0c60 WARN decodebin
gstdecodebin2.c:3705:gst_decode_bin_expose:<decodebin0> error: no suitable
plugins found
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Your
GStreamer installation is missing a plug-in.
Additional debug info:
gstdecodebin2.c(3705): gst_decode_bin_expose ():
/GstPipeline:pipeline0/GstDecodeBin:decodebin0:
no suitable plugins found
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
So it looks like decodebin cannot recognise video/x-gst-fourcc-L565 (which
is RGB16, I believe).
Any suggestions?
Dan
More information about the gstreamer-devel
mailing list