Using appsrc with infrequent buffers

[Adam Goodwin]

Thanks a lot, I didn't know about gap events - I'm looking into generating
these periodically now. I'm thinking of probably using
g_timeoout_add_seconds() and getting the callback to call
gst_element_send_event() for the appsrc, hopefully this is an acceptable
approach.


More interestingly, I've just tried something else though. I thought about
what you said, but if the problem is in the muxer, then I couldn't work out
why my pipeline's RTP branch is being held up by the appsrc being connected
to the muxer. With all the queues I have, I would have thought any hangups
would be restricted to just the appsrc and muxer.

But while investigating the matroskamux anyway, I think I've found the
problem, Instead of even having the appsrc connected to the muxer, I've
changed the pipeline so that the matroskamux only has one sink pad (for the
video). I also have the appsrc still, but I've connected it to its queue
element and then a fakesink (instead of the queue then the matroskamux).

What I'm seeing is the same problem. The receiver application looks to not
be getting anything from over RTP, and the MKV file still has most of the
video missing along with other glitches. The appsrc simply being present in
the pipeline is enough to mess things up.

However, manually giving the appsrc data alleviates the problem as before.
I now realise that there isn't anything wrong with the RTP section of the
pipeline. What seems to be happening is that when the program starts, the
keyframe from the camera (i.e. the  H.264 I-frame) is produced before the
appsrc pushes any buffers. It seems that the appsrc holds up the entire
flow of the pipeline somehow, so that the video stream - up until the first
appsrc push - is lost. So the receiver doesn't start displaying the RTP
stream until the next I-frame comes through which can take some time.

Going back to my original pipeline, with the appsrc connected to a subtitle
pad of the matroskamux, I was able to produce a reasonable MKV file which
VLC could play - including with my subtitles (albeit with a few hiccups of
missing video still). All I had to do to get the better result was feed in
a line of text to the appsrc quickly enough after the application started
running. I also had to keep feeding lines into the application from stdin,
without too much time between each line, as leaving too long a gap seems to
be when things start to go really wrong.

So the root problem seems to be that if appsrc goes too long without
pushing a buffer, the whole flow of data in the pipeline is stopped. Even
with all my branches having queues which I thought would avoid this sort of
global hangup.

So absolutely it seems like your fix of gap events will work, or if not I
can just make sure to give some nonsense subtitle buffers to the appsrc
instead of generating events. In particular I'll have to make sure a buffer
is pushed the instant the pipeline starts to play, so as not to lose the
first video buffers. But my real question now is if this behaviour of
appsrc is a bug? It certainly doesn't seem like appsrc should be able to
hold up the whole pipeline whenever it hasn't had any buffers for a while -
especially when all my branches have queues in order to avoid this type of
problem. But then again, it's possible that there's something I have
configured wrong in my application. Any ideas on this?

Thanks


On Wed, Sep 11, 2013 at 3:33 AM, Tim-Philipp Müller <t.i.m at zen.co.uk> wrote:

> On Wed, 2013-09-11 at 03:10 +1200, Adam Goodwin wrote:
>
> Hi,
>
> > (sorry to keep asking for help so much :/).
>
> Not a problem, that's what the list is for!
>
>
> > I've still got my H264 streams going from cameras into a matroskamux
> > (and one stream being sent over RTP). At the moment I'm working with
> > only one v4l2src, because this seems to work stably for me, so there's
> > only one video stream.
> >
> >
> > I've now added an appsrc which I've hooked up to the a subtitle pad on
> > the matroskamux. I haven't put together the subtitle retrieval at the
> > playback side of things, but the appsrc seems to be working when I
> > push buffers (none of my error checks come back with anything).
> >
> >
> > However:
> >
> >
> > My video doesn't seem to be sent over RTP unless I type a line of text
> > into my application - which causes the appsrc to push a buffer (with
> > the text forming a subtitle). Once I've typed a line, I start to see
> > video in my RTP receiver application. However, it quickly comes to a
> > stop again until I type more. So it appears the pipeline doesn't like
> > it when appsrc isn't contributing.
> >
> >
> > As well as this, if I play my resulting MKV in VLC, the playback is
> > completely messed up. Whole sections of video are missing (or maybe
> > sped up so much that they're essentially skipped), and the progress
> > bar skips around all over the place. Despite this, I did see my appsrc
> > subtitles displayed by VLC a few times. So it suggests to me that
> > there are timing issues in the pipeline, and that it's not necessarily
> > anything to do with the appsrc related stuff (although it was adding
> > the appsrc that triggered these issues, which are somewhat similar to
> > those I was having when a second camera is introduced). Is it possible
> > that the matroskamux is having issues with multiple input streams?
>
> The problem is likely in the muxer (or rather the GstCollectPads
> functionality it uses internally). It might wait for a buffer on all
> pads before continuing / doing something.
>
> You might be able to work around this by pushing a GAP event on the
> subtitle appsrc whenever you know that there won't be any subtitle data,
> or when there hasn't been any sub data for the last second or so. That
> should hopefully keep the muxer ticking along.
>
> Cheers
>  -Tim
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130911/34591ba5/attachment-0001.html>