Audiomixer dropping input when mixing live and non-live sources

[Nicolas Dufresne]

Le 12 mars 2017 4:35 AM, "Sean DuBois" <sean at siobud.com> a écrit :

Hey list!

I am attempting to combine a mixture of live and non-live sources, however
I am having trouble
with the audiomixer dropping audio. The following is my example pipeline,
however the audio is lost from my rtmpsrc.
The rtmpsrc is 'live' it is a h264/aac FLV and is is produced from a remote
camera on the fly.


```
#include <gst/gst.h>

int main(int argc, char *argv[]) {
  gst_init(&argc, &argv);

  auto *loop = g_main_loop_new(NULL, FALSE);
  auto *pipeline = gst_parse_launch(
      "videotestsrc is-live=true ! compositor name=c !
video/x-raw,width=1280,height=720 ! queue ! autovideosink "
      "audiotestsrc volume=0.0 is-live=true ! audiomixer name=a ! queue !
autoaudiosink "
      "rtmpsrc location=\"rtmp://localhost/serve/live\" ! decodebin name=d
! videoconvert name=vconv ! queue ! c. d. ! audioconvert name=aconv ! queue
! a.",
      NULL);

  gst_element_set_state(pipeline, GST_STATE_PLAYING);
  g_main_loop_run(loop);

  return 0;
}
````

If I remove `is-live=true` from the videotestsrc and audiotestsrc the audio
works.
If I add latency=2000000000 to the compositor/audiomixer the audio works.

However, I can't add the latency attribute because other srcs on the
audiomixer/compositor (rtp) break things very quickly


Can you clarify how it fails for you? You need some latency for this to
work, but 2s might bee to much. You would need enough latency on rtp jitter
buffer too.


One thing I do find peculiar is that the compositor always works it is just
empty, there is some difference in logic/state
between the audiomixer/compositor (where the compositor behavior is the
well behaving one)


Video is simpler to deal with, since you can repeat frames without you
noticing.


I also can do a GST_PAD_PROBE_BUFFER and add ~2 seconds to the PTS of the
raw audio buffers on the audioconvert sink pad, and that fixes it as well.
However I don't understand where that 2 second of loss is coming from? I
would like to measure/understand, before I do a hack
like that.

So if anyone has any ideas/can point out what I am doing wrong I would love
to hear!

thanks
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170312/50242589/attachment.html>