problem with mpegtsmux for v4l2src and pulsesrc

[Nicolas Dufresne]

Le lundi 21 juin 2021 à 20:33 +0430, Ali G. Dezfuli via gstreamer-devel a
écrit :
> Hi all,
> I have just got familiar with gstreamer and found it great and started to learn
> it as fast as I can.
> I downloaded some tutorials from here and there and also studied the official
> tutorial of the webpage.
> BUT ...
> I couldn't find a clear example of multiplexing video and audio using mpegtsmux.
> To be clear, I am using ubuntu 16.04 and the following line always shows error:

Hi, this pipeline has multiple issues, some of simple, but some are more
complex.

> 
> $ gst-launch-1.0 -e -v mpegtsmux name=m ! filesink location=/tmp/multi.ts     v4l2src ! videoconvert ! x264enc ! m.      pulsesrc ! audioconvert ! lamemp3enc  ! m.

- First thing first, you need h264parse after x264enc.
- You also need mpegaudioparse after lamemp3enc.

The should resolve the not-neogitated error. Now, that pipeline stalls. The
reason is that there is a latency issue. At 10fps, x264 in its default requires
5.5s of latency (55 frames), but v4l2src can only carry 32 frames before
starting to loose frame. You might be tempted to just add a queue after v4l2src,
but v4l2src is limited by the Linux kernel to 32 buffers and will refuse to let
the max latency go higher. So we have to reduce x264enc latency. Here's a
configuration that do reduce the latency, you could also use tune=zerolatency to
get no latency, this is at a quality cost.

   x264enc rc-lookahead=3 threads=2 

Now, the pipeline still complain, since the audio branch also needs hold on that
latency, fortunatly there, you can add a queue after pulsesrc. Just to give it
more room, I have also increased the pulsesrc buffer-time. So a working pipeline
(for my setup) would be:

gst-launch-1.0 -e -v mpegtsmux name=m ! filesink location=/tmp/multi.ts \
    v4l2src ! videoconvert ! x264enc rc-lookahead=3 threads=2 ! h264parse ! m. \
    pulsesrc buffer-time=2000000 ! queue ! audioconvert ! lamemp3enc  ! mpegaudioparse ! m.

With hope this helps. This was worked on from the reported latency errors, and
with the latency tracer. See:

https://gstreamer.freedesktop.org/documentation/coretracers/latency.html?gi-language=c
https://gstconf.ubicast.tv/videos/lightning-talks_93212/ (time 32:00)

May not work great on old GStreamer version like yours.

> 
> I would be grateful if anybody could help me with that.
> regards,
> Ali
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel