How to mix live and non-live sources
Marianna Smidth Buschle
msb at qtec.com
Fri Oct 1 07:40:36 UTC 2021
Thanks for the help
I have set the offset on the src pads of the decoders (for both audio
and video) and I got some progress but still have some issues.
From what I can see it relies on the Segment event? I seem to see a new
segment (with a base equal to the offset value) event when I set the pad
So I think the problems I still have might be on the pipeline decoupling
element (interpipesrc/sink) I'm using not handling the Segment correctly.
I have founds some problems in their code and corrected that, but might
still be missing something.
Specifically my problem is like this: I'm listening to the live source
but want to change to a replay sequence.
The replay sequence is a short transition video, then the replay stream
itself, followed by the transition video again and then back to live.
A simplification of the topology:
v4l2src -> interpipesink name=vlive
pulsesrc -> interpipesink name=alive
filesrc location=transition.ts -> tsdemux -> h264_dec -> interpipesink
-> acc_dec -> interpipesink
filesrc location=cam.ts -> tsdemux -> h264_dec -> interpipesink name=vreplay
-> aac_dec -> interpipesink name=areplay
interpipesrc name=vselect listen-to=vlive -> h264_enc -> tsmux ->
rtpmp2tpay -> udpsink
interpipesrc name=aselect listen-to=alive -> aac_enc ->
So when I want to do a replay I start by preparing the replay file
pipeline stream by playing, seeking and then pausing (because I found
out that seeking takes time and I need seamless changes).
Then I play the transition file pipeline, set the pad offset on the
decoders::src and set the interpipesrc to listen-to vtranstion/atransition.
When I get the EOS I play the replay file pipeline, set the pad offset
on the decoders::src and set the interpipesrc to listen-to vreplay/areplay.
When I get next the EOS I set the transition file pipeline to NULL, then
to PLAYING, set the pad offset on the decoders::src and set the
interpipesrc to listen-to vtranstion/atransition.
On this final EOS I just set the interpipesrc to listen-to vlive/alive
And the problem is that I see a new segment when I play the transition
file pipeline the 1st time and also when I play the replay file
pipeline, but not when I play the transition file pipeline the 2nd time.
So the tsmux accepts the frames of the 1st transition and the replay
itself, but complains about backwards DTS for the end transition
(because there is no new Segment with a proper offset).
So, is there something wrong going on the interpipe, or do I need to
"reset" the transition file pipeline on a different manner than setting
to NULL in order to be able to get the new Segment?
On 24.09.2021 13.23, Sebastian Dröge wrote:
> On Fri, 2021-09-24 at 13:17 +0200, Marianna Smidth Buschle wrote:
>> On 24.09.2021 13.05, Sebastian Dröge wrote:
>> On Fri, 2021-09-24 at 12:59 +0200, Marianna Smidth Buschle via
>> gstreamer-devel wrote:
>>> I have a use case where I have N live sources (v4l2src) from which I
>>> select to be the output.
>>> I also need to be able to make a replay (seek from a filesrc).
>>> My question is how to properly handle the mixing of the live and
>>> non-live sources since the timestamps don't match.
>>> The filesrc starts from zero (or the seeking point) and the live
>>> have a running-time based on the clock.
>> This is actually simpler than what you wrote. What you would do is to
>> call gst_pad_set_offset() on the source pad of the non-live streams
>> that you want to connect to the mixer.
>> That would then make sure to update the running time accordingly.
>> Can I do that directly on the filesrc (the MPEG-TS) stream?
>> Or should it be on the tsdemux pads, or later on the audio/video
>> decoder pads?
> filesrc doesn't know anything about time, so that won't have any effect.
> I would do it directly on the source pads that you connect to
> Sebastian Dröge, Centricular Ltd · https://www.centricular.com
Best regards / Med venlig hilsen
“Marianna Smidth Buschle”
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the gstreamer-devel