Muxing videos with 'sparse' audio streams using mpetgsmux

[Ben Muzal]

Hello,

I have a somewhat unusual issue and am looking for advise on the best way
to proceed. I am using gstreamer 1.14, and I am familiar with gstreamer
development enough to have made my own plugin or two and I already deploy a
custom package of gst-plugins-bad to handle some poorly labeled metadata
streams in MPEG-TS files.

The next task I have is to take a live MPEG-TS stream, demux it, do some
modifications to the metadata stream, then remux it as MPEG-TS again, and
finally split it into files using splitmuxsink to get files that start on
good keyframe boundaries where possible.

The issue is that a significant number of the input streams have an audio
stream listed in the PMT, but don't actually have any audio or only have
audio every once in a while. Basically the audio stream is sparse. This is
not to say that there is silent audio, instead there are no packets at all
on the PID that claims to be audio.  Also, there is not a way to tell if
there is not going to be audio until I don't get it. I wish I had some sort
of control over the input streams, but I do not.

Because audio streams are never marked as sparse,  GstCollectPads in
mpetgsmux will wait forever for the first packet of an audio stream which
may never get there. This locks up the pipeline.

I figure that I either need to modify tsdemux to automatically detect and
mark the misbehaved  audio streams as sparse if they don't have packets
after a while, or update GstCollectPads to have a mode that allows waiting
pads to time out. There may be other options that I have not considered as
well.

What is the most logical way to proceed within the gstreamer framework that
is not going to cause unexpected behavior elsewhere? It seems that tsdemux
being honest about if a stream is sparse or not makes initial sense, but
will downstream elements (like muxers) be able to handle getting delayed
codec information?

Thank you.
--Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190515/440b8bbb/attachment-0001.html>