Audiomixer dropping input when mixing live and non-live sources

Sean DuBois sean at siobud.com
Sun Mar 12 08:22:20 UTC 2017


Hey list!

I am attempting to combine a mixture of live and non-live sources, however I am having trouble
with the audiomixer dropping audio. The following is my example pipeline, however the audio is lost from my rtmpsrc.
The rtmpsrc is 'live' it is a h264/aac FLV and is is produced from a remote camera on the fly.

```
#include <gst/gst.h>

int main(int argc, char *argv[]) {
  gst_init(&argc, &argv);

  auto *loop = g_main_loop_new(NULL, FALSE);
  auto *pipeline = gst_parse_launch(
      "videotestsrc is-live=true ! compositor name=c ! video/x-raw,width=1280,height=720 ! queue ! autovideosink "
      "audiotestsrc volume=0.0 is-live=true ! audiomixer name=a ! queue ! autoaudiosink "
      "rtmpsrc location=\"rtmp://localhost/serve/live\" ! decodebin name=d ! videoconvert name=vconv ! queue ! c. d. ! audioconvert name=aconv ! queue ! a.",
      NULL);

  gst_element_set_state(pipeline, GST_STATE_PLAYING);
  g_main_loop_run(loop);

  return 0;
}
````

If I remove `is-live=true` from the videotestsrc and audiotestsrc the audio works.
If I add latency=2000000000 to the compositor/audiomixer the audio works.

However, I can't add the latency attribute because other srcs on the audiomixer/compositor (rtp) break things very quickly

One thing I do find peculiar is that the compositor always works it is just empty, there is some difference in logic/state
between the audiomixer/compositor (where the compositor behavior is the well behaving one)

I also can do a GST_PAD_PROBE_BUFFER and add ~2 seconds to the PTS of the raw audio buffers on the audioconvert sink pad, and that fixes it as well.
However I don't understand where that 2 second of loss is coming from? I would like to measure/understand, before I do a hack
like that.

So if anyone has any ideas/can point out what I am doing wrong I would love to hear!

thanks


More information about the gstreamer-devel mailing list