mpegtsmux with ximagesrc

[Tim-Philipp Müller]

On Mon, 2011-07-25 at 10:19 -0700, Cahill, Ben M wrote:

> I'm trying to capture desktop display (via H264) and system audio (via
> AAC) to send via TS->RTP->udp to another computer.
> 
> The pipeline below works as long as something is moving within the
> desktop display.  But, if display doesn't change, ximagesrc stops
> delivering frames (which saves needless thrash), and mpegtsmux output
> stops dead; mpegtsmux's collectpads won't do anything until *all* input
> pads (i.e. both audio *and* video) have data queued.
> 
> Is this "sporadic", non-periodic video a valid usage case for TS?
> 
> If so, any suggestions for an alternative/workaround?  I've tried
> videorate in front of the H264 encoder, but that swamps the system with
> needless thrash.  I'm also trying to hack mpegtsmux to allow it to
> package/forward audio frames even when video stops, but having problems
> getting things started (non-negotiated caps?? for the stream that
> starts second).

There are known issues with GstCollectPads and handling of sparse
streams, see for example [1], but there are probably ways this could be
hacked around until something better comes along.

Some random ideas: does ximagesrc send newsegment updates, to move time
along when it doesn't output repeat frames? (Then the muxer/collectpads
know when not to wait, but see above). Or maybe there's a way to tell
x264enc that we want a repeat frame and/or that the frame being passed
is identical to the previous frame? Then videorate could flag duplicates
it creates and x264enc could pick up on the flag.

 Cheers
  -Tim

[1] https://bugzilla.gnome.org/show_bug.cgi?id=415754