Overlay on a live source

[Matthew Waters]

Hi,

On 18/10/14 20:33, Moore, Thomas (FtWorth) wrote:
> I tried videorate without success. The following bug appears to confirm that videorate does not duplicate frames of a live source:
>
> https://bugzilla.gnome.org/show_bug.cgi?idr0104
>
> Thomas
> ________________________________________
> From: gstreamer-devel [gstreamer-devel-bounces at lists.freedesktop.org] on behalf of Paul Barber [paul.barber at oncology.ox.ac.uk]
> Sent: Saturday, October 18, 2014 2:41 AM
> To: Discussion of the development of and with GStreamer
> Subject: Re: Overlay on a live source
>
> Hi,
> I have a similar problem but have not really solved it either. Can you use a videorate element to maintain a constant fast frame rate (by duplicating frames) from a source that may stall or run at a reduced rate for some reason?
> Cheers,
> Paul
>
> On 17 October 2014 22:20:02 BST, "Moore, Thomas (FtWorth)" <Thomas.Moore2 at ATK.COM> wrote:
> Hello,
>
>
> I have an interesting problem that I hope you can help me with. I’m working on an application that will overlay custom graphics onto a live video stream in real time. Ultimately, I’m planning on using a udpsrc and the appropriate decoders to process the video stream and cairooverlay to generate the custom graphics (we already have the cairo part done from a previous project, so that part is easy). I’ve been experimenting with several different combinations of plugins to achieve the desired effect, but I’m running into two problems.
>
>
> First, the pipeline isn’t constructed until the udpsrc receives data. I think this can be overcome by using a videotestsrc in place of the udpsrc (and decoders) and a videoswitch to swap to the video stream once data is detected via a filter pad.
>
>
> Second, if the video stream stops (or pauses), then so does the output of the pipeline. At first, I just had the cairooverlay on top of the uspsrc/decoder chain, and it makes sense that the pipeline will stop if the streaming video stops. My next thought was to use a video mixer to mix the cairooverlay atop a transparent videotestsrc with the live video stream. This approach sort of works, but with a live stream as an input to the video mixer, the pipeline still seems to stop if the video stream stops. Here is an example pipeline that demonstrates the issue:
>
>
> gst-launch-1.0 -v videotestsrc name=tsrc0 is-live=true ! \
>                   video/x-raw,width80,heightr0,framerate=1/1 ! \
>                   videomixer name=x ! \
>                   videoconvert ! \
>                   ximagesink syncúlse \
>                   videotestsrc name=tsrc1 patternºll background-color=0 ! \
>                   video/x-raw,format=GB,width80,heightr0,framerate/1 ! \
>                   mix.

That's correct, videomixer will stop producing data if it does not
receive input frames from any pad.  There is recent work on master [1]
to fix this using GstAggregator of which the compositor element is based
on.  Play around with the timeout property on compositor which should be
at least the length of time for one buffer to appear on a single
compositor sink pad.

[1]
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=a41bc98b6e7436e5bb4a465b55674f47a98a6f5f

> You’ll notice that tstsrc0, which is the underlay, is running at 1 fps and tstsrc1, the overlay, is running at 10 fps. If is-live is set to false on tstsrc0, then the pipeline runs as expected with the ball moving at 10 fps and the underlay updating at 1 fps. However, when is-live is set to true (which requires the syncúlse on the sink to prevent timing errors), the ball moves for about half a second and then stops for about half a second and then repeats.
>
>
> For these tests, I originally started with GStreamer 0.10 as installed with openSUSE 12.1, but I have since updated to openSUSE 13.1 and am running the following versions:
>
>
> gstreamer 1.4.3-3.2
> gstreamer-plugins-base 1.4.3-2.4
> gstreamer-plugins-good 1.4.3-2.1
> gstreamer-plugins-bad 1.4.3-5.7
> gstreamer-plugins-bad-orig-addon 1.4.3-5.7
> gstreamer-plugins-ugly 1.4.3-2.2
> gstreamer-plugins-ugly-orig-addon 1.4.3-2.2
> gstreamer-plugins-libav 1.4.3-1.1
>
>
> Any ideas as to how I can get this working would be greatly appreciated.
>
>
> Thanks!
>
>
> Thomas
>
>
>
>
>
>
>
>
>
>
>
>
>
> ________________________________
>
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20141020/b474d9ef/attachment.sig>