[Bug 772766] New: V4L2 compositor proposal
GStreamer (GNOME Bugzilla)
bugzilla at gnome.org
Tue Oct 11 16:03:44 UTC 2016
https://bugzilla.gnome.org/show_bug.cgi?id=772766
Bug ID: 772766
Summary: V4L2 compositor proposal
Classification: Platform
Product: GStreamer
Version: git master
OS: Linux
Status: NEW
Severity: enhancement
Priority: Normal
Component: gst-plugins-good
Assignee: gstreamer-bugs at lists.freedesktop.org
Reporter: sebastien.matz at veo-labs.com
QA Contact: gstreamer-bugs at lists.freedesktop.org
GNOME version: ---
Created attachment 337446
--> https://bugzilla.gnome.org/attachment.cgi?id=337446&action=edit
V4L2 compositor code
Hi folks,
Here is a proposal of a new GStreamer element which performs
video composition using video4linux2 M2M devices.
Today we are able to use our new element with simple pipelines using
gst-launch and also more complex ones using our application on our
platform (imx6 board). In both cases, fbdevsink element (with sometimes
others sinks) is used to display the composition's result. Used capture/output
resolution is 720p or 1080p and we use various video format: YUV2, NV12 and
RGB16. Ouput IO-mode is always dmabuf-import and capture IO-mode is always
dmabuf.
The current status of our development is work-in-progress and well
there is still some bugs. It seems to work as expected with
gst-launch but we experiment different errors when using it in
our application after some start/stop cycles (== gstreamer pipeline
reinstanciation from a same process). That is why life cycle of our buffers is
greatly suspected. Typically encountered errors are after a failing start are:
ERROR v4l2allocator
gstv4l2allocator.c:1245:gst_v4l2_allocator_qbuf:<ipu1-capture:pool:src:allocator>
failed queueing buffer 0: Invalid argument
ERROR v4l2bufferpool
gstv4l2bufferpool.c:1127:gst_v4l2_buffer_pool_qbuf:<ipu1-capture:pool:src>
could not queue a buffer 0
ERROR v4l2allocator
gstv4l2allocator.c:1245:gst_v4l2_allocator_qbuf:<ipu1-capture:pool:src:allocator>
failed queueing buffer 1: Invalid argument
ERROR v4l2bufferpool
gstv4l2bufferpool.c:1127:gst_v4l2_buffer_pool_qbuf:<ipu1-capture:pool:src>
could not queue a buffer 1
(...)
*or*
ERROR gst_v4l2_compositor_queue_jobs() failed
which means buffers are still owned somewhere in the pipeline (downstream).
*or*
process freezing :(
A this point I cannot find where is/are the problem(s). We also use
non-standard elements in our application or plaform (gstreamer elements,
drivers, ...). So at this point some help/feedback should be appreciated.
Can someone try to run and compile it on another platform ?
Is there an easy way to restart the pipeline from gst-launch (gstreamer
pipeline
reinstanciation from a same process ?)
I put two patches in attachment. Both should apply on the current master
branch.
First one (v4l2compositor.patch) is the code of the compositor itself. Note
that we use gstvideoaggregator (from gst-plugins-bad) as base class after
importing and renaming it in the gst-plugins-good repository.
Second one (testing_with_gst-launch.patch) is only for testing purpose only. It
makes some modifications in v4l2convert that we put in front of fbdevsink in
some of our gst-launch commands (see below).
Testing v4l2compositor with gst-launch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is a complete example with gst-launch with some logs. It composes two
stream into one. Here all streams are 720p:
# GST_DEBUG_NO_COLOR=1 GST_DEBUG=*:3 gst-launch-1.0 -v v4l2compositor
num_jobs=12 device=/dev/v4l/by-path/ipu1-scaler output-io-mode=5
capture-io-mode=4 name=comp sink_0::xpos=8 sink_0::ypos=8 sink_0::width=624 sin
k_0::height=344 sink_1::xpos=648 sink_1::ypos=8 sink_1::width=624
sink_1::height =344 !
"video/x-raw,pixel-aspect-ratio=1/1,colorimetry=bt709,height=720,framer
ate=24/1,width=1280" ! v4l2convert device=/dev/v4l/by-path/ipu2-scaler
output-io-mode=5 capture-io-mode=auto ! fbdevsink v4l2src
device=/dev/v4l/by-path/ipu1-capture io-mode=4 !
"video/x-raw,pixel-aspect-ratio=1/1,colorimetry=bt709,height=720,framerate=24/1,width=1280,format=YUY2"
! comp. v4l2src device=/dev/v4l/by-path/ipu2-capture io-mode=4 !
"video/x-raw,pixel-aspect-ratio=1/1,colorimetry=bt709,height=720,framerate=24/1,width=1280,format=YUY2"
! comp.
Setting pipeline to PAUSED ...
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2
gstv4l2object.c:1849:gst_v4l2_object_add_aspect_ratio:<v4l2transform0> Failed
to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
WARN v4l2src gstv4l2src.c:545:gst_v4l2src_query:<v4l2src0>
Can't give latency since framerate isn't fixated !
Pipeline is live and does not need PREROLL ...
WARN v4l2src gstv4l2src.c:545:gst_v4l2src_query:<v4l2src1>
Can't give latency since framerate isn't fixated !
WARN v4l2_aggregator
gstv4l2aggregator.c:1409:gst_v4l2_aggregator_query_latency_unlocked:<comp>
Latency query failed
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw,
format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstV4l2Compositor:comp.GstV4l2CompositorPad:sink_0: caps
= video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw,
format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstV4l2Compositor:comp.GstV4l2CompositorPad:sink_1: caps
= video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, framerate=(fraction)24/1
WARN v4l2bufferpool
gstv4l2bufferpool.c:748:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src>
Uncertain or not enough buffers, enabling copy threshold
WARN v4l2bufferpool
gstv4l2bufferpool.c:748:gst_v4l2_buffer_pool_start:<v4l2src1:pool:src>
Uncertain or not enough buffers, enabling copy threshold
FIXME basesink
gstbasesink.c:3124:gst_base_sink_default_event:<fbdevsink0> stream-start event
without group-id. Consider implementing group-id handling in the upstream
elements
WARN structure
gststructure.c:1935:priv_gst_structure_append_to_gstring: No value transform to
serialize field 'chroma-site' of type 'string'
/GstPipeline:pipeline0/GstV4l2Compositor:comp.GstPad:src: caps = video/x-raw,
format=(string)YUY2, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)bt709, interlace-mode=(string)progressive,
chroma-site=(string)NULL
WARN structure
gststructure.c:1935:priv_gst_structure_append_to_gstring: No value transform to
serialize field 'chroma-site' of type 'string'
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)bt709, interlace-mode=(string)progressive,
chroma-site=(string)NULL
/GstPipeline:pipeline0/GstV4l2Transform:v4l2transform0.GstPad:src: caps =
video/x-raw, format=(string)RGB16, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFBDEVSink:fbdevsink0.GstPad:sink: caps = video/x-raw,
format=(string)RGB16, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive
WARN v4l2bufferpool
gstv4l2bufferpool.c:748:gst_v4l2_buffer_pool_start:<v4l2transform0:pool:src>
Uncertain or not enough buffers, enabling copy threshold
WARN structure
gststructure.c:1935:priv_gst_structure_append_to_gstring: No value transform to
serialize field 'chroma-site' of type 'string'
/GstPipeline:pipeline0/GstV4l2Transform:v4l2transform0.GstPad:sink: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)bt709, interlace-mode=(string)progressive,
chroma-site=(string)NULL
WARN structure
gststructure.c:1935:priv_gst_structure_append_to_gstring: No value transform to
serialize field 'chroma-site' of type 'string'
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720,
framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)bt709, interlace-mode=(string)progressive,
chroma-site=(string)NULL
Redistribute latency...
<Ctrl+C>
Execution ended after 0:00:36.615017671
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Regards,
Sebastien
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
More information about the gstreamer-bugs
mailing list