a/v freeze while flushing audio chain of pipeline in running state (clock and renderer sync related)

[Enrique Ocaña González]

El jueves, 8 de junio de 2017 14:18:51 (CEST) Nitin Mahajan escribió:

> The sequence of operations done on adaptive demux plugin are as follows:
> On audio src pad of demux:
> Flush buffered audio packets:
> - push gst_event_new_flush_start()
> - push gst_event_new_flush_stop(FALSE)
> - push new segment event
> - push buffers with the time stamps corresponding to the current
> presentation time

First of all, sorry if my comment isn't completely related to the problem you 
describe. Feel free to ignore it if it only adds noise to the discussion. I 
think it might be relevant, though.

I'm having a similar problem in our use case: Two appsrcs pushing already 
demuxed buffers (video/x-h264 and audio/mpeg) and the pipeline filled of 
buffers. Then our app receives an update of some of the audio buffers (eg: 
higher/lower quality) and needs to replace the buffers already in the pipeline 
(playbin) in a seamless way (keeping the streamsynchronizer happy and the 
video stream unaware of the replacement in the audio stream).

We do something similar to what you describe:

- We ask the pipeline for the current position (the PTS of the buffer being 
rendered)
- We send flush start and stop (false) events
- Use gst_base_src_new_seamless_segment() to send a new [position, old_stop] 
  segment (to convince appsrc to schedule the new buffers properly)
- Fix the segment emitted by appsrc using a probe, so it's again [position, 
  old_stop] and clear flags (not reset-time).

The new replacement buffers start to fill the audio stream of the pipeline 
again, the faad audio decoder does fine (thanks to the flush), and the audio 
buffers reach the audio sink (GstOMXHdmiAudioSink). So far so good.

Then the problems start. For some reason, audiobasesink doesn't reconfigure 
its audioringbuffer properly. Normally, I have a diff of ~40 between segdone 
(the write position in the audioringbuffer) and readseg (the read position of 
the data that goes to the actual audio device). That matches with segtotal 
(the size of the audioringbuffer). However, after the flushes and new segment 
the diff is ~15000 (clearly wrong, much higher than segtotal). This causes 
gst_audio_ring_buffer_commit() (implemented by default_commit) to wait until 
the diff is again less than segtotal. It waits forever.

This long audio sink wait stalls the audio stream and we start to lose audio.

If you want to know if you're having the same problem as we have, enable these 
logs:

 GST_DEBUG='basesink:TRACE,ringbuffer:TRACE,audiosink:TRACE,\
           audiobasesink:TRACE,audioclock:DEBUG'

and grep for this kind of messages:

 cat /tmp/log.txt | grep default_commit | grep 'diff'

You should see the diff jump (from ~40 to ~15000) like this:

0:00:25.508332021  1277 0x72e0d4f0 DEBUG             ringbuffer 
gstaudioringbuffer.c:1525:default_commit:<audiosinkringbuffer0> pointer at 
1033, write to 1072-1340, diff 39, segtotal 40, segsize 1764, base 0
0:00:25.508480042  1277 0x72e0d4f0 DEBUG             ringbuffer 
gstaudioringbuffer.c:1525:default_commit:<audiosinkringbuffer0> pointer at 
1033, write to 1073-0, diff 40, segtotal 40, segsize 1764, base 0
0:00:25.615070250  1277 0x72e0d4f0 DEBUG             ringbuffer 
gstaudioringbuffer.c:1525:default_commit:<audiosinkringbuffer0> pointer at 
1034, write to 16497-1204, diff 15463, segtotal 40, segsize 1764, base 0
0:00:25.615523479  1277 0x72e0d4f0 DEBUG             ringbuffer 
gstaudioringbuffer.c:1525:default_commit:<audiosinkringbuffer0> pointer at 
1035, write to 16497-1204, diff 15462, segtotal 40, segsize 1764, base 0

I haven't found a solution yet and by now we're just applying a workaround on 
the app side to avoid the need of replacing already buffered audio data.

Cheers.

-- 
Enrique Ocaña González