Mixing lossy live sources

[Benjamin Le Bozec]

Ok guys, thanks for the heads-up !

I managed to have something working using the "identity sync=1" trick, and
adding a latency to the compositor. Here is the pipeline :

gst-launch-1.0 -e rtmpsrc location="rtmp://..." ! identity sync=1 !
queue max-size-time=2000000000 ! decodebin ! videorate !
'video/x-raw,format=(string)I420,framerate=30/1,
width=1280, height=720' ! mix.sink_0 \
rtmpsrc location="rtmp://..." ! identity sync=1 ! \
queue max-size-time=2000000000 ! decodebin ! videoscale ! videorate !
'video/x-raw,format=(string)I420,framerate=30/1,
width=320, height=240' ! mix.sink_1 \
compositor name=mix latency=1000000000 ! \
videoconvert ! x264enc threads=0 bitrate=2000 tune=zerolatency
key-int-max=60 speed-preset=veryfast ! \
flvmux streamable=true name=mux ! rtmpsink sync=1 location="rtmp://...
live=1"

When one of the inputs starts to lag, the compositor continues to produce
images. Amazing :)

The only thing i'm not confortable with is the arbitrary latency set to
1sec.

I also managed to mix the audio, but i spare you the pipeline ...

Thank you,

Ben

2018-05-17 17:21 GMT+02:00 Nicolas Dufresne <nicolas at ndufresne.ca>:

> Le jeudi 17 mai 2018 à 12:52 +0100, Tim Müller a écrit :
> > On Thu, 2018-05-17 at 12:28 +0200, Benjamin Le Bozec wrote:
> >
> > Hi Benjamin,
> >
> > > I'm mixing two RTMP live sources with the compositor element :
> > > rtmpsrc (do-imestamp=true) ==>
> > >                                                        compositor
> > > ==>
> > > rtmpsink (sync=true)
> > > rtmpsrc (do-imestamp=true)  ==>
> > >
> > > But the sources are coming from cellphones, through a lossy network
> > > (the cellular network), and packets are lost from time to time.
> > > Consequently, the compositor elements tends to freeze, waiting for
> > > both streams to deliver packets.
> > >
> > > Is there a way to ensure a continuous streaming, that is for the
> > > compositor to continue to deliver images even if some (or all)
> > > sources do not deliver images ?
> >
> > compositor *should* be able to handle this scenario well. However, it
> > will do so only if the source elements have identified themselves to
> > be
> > 'live' sources (like rtspsrc/udpsrc). I don't think rtmpsrc does
> > that.
> >
> > You *might* be able to hack around it by using
> > gst_base_src_set_live()
> > and setting the "latency" property on compositor to some reasonable
> > value; or add a pad probe and intercept the latency query from
> > compositor to the sources.
>
> I believe you can turn this into a lice source using the following
> construction, let me know if that works for you.
>
>   rmptsrc do-timestamp=1 ! identity sync=1 ! ...
>
> You can place the identity elsewhere if needed.
>
> >
> > Not sure if rtmpsrc will gracefully handle this case though (drop
> > packets to 'catch up').
> >
> > Cheers
> >  -Tim
> >
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>


-- 

*Benjamin Le Bozec*

Tél. +33 (0)2 99 63 42 36
*http://www.vodalys.com* <http://www.vodalys.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180518/121c3895/attachment.html>