mpegtsmux with ximagesrc

[Cahill, Ben M]

Update to this situation of audio blocked by sparse video in mpegtsmux ... after some IRC discussion in late July, I've posted some patches at:

https://bugzilla.gnome.org/show_bug.cgi?id=415754

These include (paraphrased):

Move GstCollectPads2 into gstreamer library
Port mpegtsmux to use GstCollectPads2
Allow sparse (non-waiting) mode for any sink pad in mpegtsmux
Use bitfield-based "collected" and "WAIT" algos in GstCollectPads2
Prototype "sparsesegment" inserter plugin

The "sparsesegment" plugin is pretty primitive; it works for my purposes (so far), but as I'm segments noobie, it may break some usage models, and will probably need some additional development.

This is all working for me (AFAICT), but I would appreciate any comments, testing, etc.

-- Ben --


>-----Original Message-----
>From: 
>gstreamer-devel-bounces+bmcahill=alum.rpi.edu at lists.freedesktop
>.org 
>[mailto:gstreamer-devel-bounces+bmcahill=alum.rpi.edu at lists.fre
edesktop.org] On Behalf Of Cahill, Ben M
>Sent: Tuesday, July 26, 2011 3:28 PM
>To: Discussion of the development of and with GStreamer
>Subject: RE: mpegtsmux with ximagesrc
>
> 
>
>>-----Original Message-----
>>From: 
>>gstreamer-devel-bounces+bmcahill=alum.rpi.edu at lists.freedesktop
>>.org 
>>[mailto:gstreamer-devel-bounces+bmcahill=alum.rpi.edu at lists.fre
>>edesktop.org] On Behalf Of Tim-Philipp Müller
>>Sent: Tuesday, July 26, 2011 4:50 AM
>>To: gstreamer-devel at lists.freedesktop.org
>>Subject: Re: mpegtsmux with ximagesrc
>>
>>On Mon, 2011-07-25 at 10:19 -0700, Cahill, Ben M wrote:
>>
>>> I'm trying to capture desktop display (via H264) and system 
>>audio (via
>>> AAC) to send via TS->RTP->udp to another computer.
>>> 
>>> The pipeline below works as long as something is moving within the
>>> desktop display.  But, if display doesn't change, ximagesrc stops
>>> delivering frames (which saves needless thrash), and 
>mpegtsmux output
>>> stops dead; mpegtsmux's collectpads won't do anything until 
>>*all* input
>>> pads (i.e. both audio *and* video) have data queued.
>>> 
>>> Is this "sporadic", non-periodic video a valid usage case for TS?
>>> 
>>> If so, any suggestions for an alternative/workaround?  I've tried
>>> videorate in front of the H264 encoder, but that swamps the 
>>system with
>>> needless thrash.  I'm also trying to hack mpegtsmux to allow it to
>>> package/forward audio frames even when video stops, but 
>>having problems
>>> getting things started (non-negotiated caps?? for the stream that
>>> starts second).
>>
>>There are known issues with GstCollectPads and handling of sparse
>>streams, see for example [1], but there are probably ways 
>this could be
>>hacked around until something better comes along.
>
>Thanks for the bugzilla reference.  In my tarball source code 
>(couple months old now) I see gstcollectpads2 is in 
>gst-plugins-good-0.10.29/gst/videomixer, and is used only by 
>videomixer2.c.
>
>Is anyone planning to port mpegtsmux to use gstcollectpads2?  
>I could try it as an experiment.
>
>Last night I got something "working" here by hacking 
>gstcollectpads.c to use a g_cond_timed_wait() instead of 
>g_cond_wait() (in GST_COLLECT_PADS_WAIT()), and some other 
>hacks to both gstcollectpads.c and mpegtsmux.c.  A 100 msec 
>timeout got things working afaict, but I wouldn't be surprised 
>if there are some undesired side effects that I wouldn't 
>understand yet (kind of newbie here).
>
>I'll guess gstcollectpads2.c provides a more well-thought-out solution.
>
>
>>
>>Some random ideas: does ximagesrc send newsegment updates, to 
>move time
>>along when it doesn't output repeat frames? (Then the 
>muxer/collectpads
>>know when not to wait, but see above). Or maybe there's a way to tell
>>x264enc that we want a repeat frame and/or that the frame being passed
>>is identical to the previous frame? Then videorate could flag 
>>duplicates
>>it creates and x264enc could pick up on the flag.
>
>Thanks.
>
>-- Ben --
>
>>
>> Cheers
>>  -Tim
>>
>>[1] https://bugzilla.gnome.org/show_bug.cgi?id=415754
>>
>>_______________________________________________
>>gstreamer-devel mailing list
>>gstreamer-devel at lists.freedesktop.org
>>http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>_______________________________________________
>gstreamer-devel mailing list
>gstreamer-devel at lists.freedesktop.org
>http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>