Overlay on a live source

[Moore, Thomas (FtWorth)]

Hello,

I have an interesting problem that I hope you can help me with. I'm working on an application that will overlay custom graphics onto a live video stream in real time. Ultimately, I'm planning on using a udpsrc and the appropriate decoders to process the video stream and cairooverlay to generate the custom graphics (we already have the cairo part done from a previous project, so that part is easy). I've been experimenting with several different combinations of plugins to achieve the desired effect, but I'm running into two problems.

First, the pipeline isn't constructed until the udpsrc receives data. I think this can be overcome by using a videotestsrc in place of the udpsrc (and decoders) and a videoswitch to swap to the video stream once data is detected via a filter pad.

Second, if the video stream stops (or pauses), then so does the output of the pipeline. At first, I just had the cairooverlay on top of the uspsrc/decoder chain, and it makes sense that the pipeline will stop if the streaming video stops. My next thought was to use a video mixer to mix the cairooverlay atop a transparent videotestsrc with the live video stream. This approach sort of works, but with a live stream as an input to the video mixer, the pipeline still seems to stop if the video stream stops. Here is an example pipeline that demonstrates the issue:

gst-launch-1.0 -v videotestsrc name=tstsrc0 is-live=true ! \
                  video/x-raw,width=1280,height=720,framerate=1/1 ! \
                  videomixer name=mix ! \
                  videoconvert ! \
                  ximagesink sync=false \
                  videotestsrc name=tstsrc1 pattern=ball background-color=0 ! \
                  video/x-raw,format=ARGB,width=1280,height=720,framerate=10/1 ! \
                  mix.

You'll notice that tstsrc0, which is the underlay, is running at 1 fps and tstsrc1, the overlay, is running at 10 fps. If is-live is set to false on tstsrc0, then the pipeline runs as expected with the ball moving at 10 fps and the underlay updating at 1 fps. However, when is-live is set to true (which requires the sync=false on the sink to prevent timing errors), the ball moves for about half a second and then stops for about half a second and then repeats.

For these tests, I originally started with GStreamer 0.10 as installed with openSUSE 12.1, but I have since updated to openSUSE 13.1 and am running the following versions:

gstreamer 1.4.3-3.2
gstreamer-plugins-base 1.4.3-2.4
gstreamer-plugins-good 1.4.3-2.1
gstreamer-plugins-bad 1.4.3-5.7
gstreamer-plugins-bad-orig-addon 1.4.3-5.7
gstreamer-plugins-ugly 1.4.3-2.2
gstreamer-plugins-ugly-orig-addon 1.4.3-2.2
gstreamer-plugins-libav 1.4.3-1.1

Any ideas as to how I can get this working would be greatly appreciated.

Thanks!

Thomas






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20141017/0ac0ae4b/attachment.html>