Two glvideomixers on a tee freeze on input caps change (Was: Re: How does a muxer deal with packet loss?)
michiel at aanmelder.nl
Wed Jun 23 08:30:17 UTC 2021
> If the muxers aren't the culprit, could it be the mixers? Those are
> aggregators, too. How do those deal with missing or late inputs?
I found something! Not sure what it means, though. I found I could
trigger the freeze (or at least, something that looks the same) by
changing the window size of a screensharing WebRTC video track.
When the video went into both the preview and main glvideomixers, the
pipeline locked up. Feeding it into only one of the mixers (didn't
matter which one) it continued just fine. In the locked up state, one of
the mixers' sinkpads/proxypads would disagree on the caps.
It looks like the same thing happens, or can happen, when switching
between the real source (1280x720, variable framerate?) and the
imagefreeze source (1920x1080 at 25fps). At least, I think that's what I'm
seeing in the attached screenshot of the dump.
Why would that happen? I found this:
"videoaggregator: delay using new caps from a sink pad until the next
buffer in the queue is taken
When caps changes while streaming, the new caps was getting processed
immediately in videoaggregator, but the next buffer in the queue that
corresponds to this new caps was not necessarily being used immediately,
which resulted sometimes in using an old buffer with new caps. Of course
there used to be a separate buffer_vinfo for mapping the buffer with its
own caps, but in compositor the GstVideoConverter was still using wrong
info and resulted in invalid reads and corrupt output.
This approach here is more safe. We delay using the new caps
until we actually select the next buffer in the queue for use.
This way we also eliminate the need for buffer_vinfo, since the
pad->info is always in sync with the format of the selected buffer."
Which sounds related, but could also be a completely innocent bug fix.
Anyone have an idea why the mixers wouldn't just carry on with the new
input geometry? Could the variable frame rate from webrtcbin be an
issue? I don't know why it's 0/1, it's supposed to be a 25fps stream as
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 247164 bytes
Desc: not available
More information about the gstreamer-devel