webrtcbin fake-live filesrc and seeking

[Philipp B]

Thanks for your hints!

I didnt consider clocksync yet, its indeed an option. I do not
understand all aspects of timing yet. However, aiming to support as
many input files as possible, and then feeding realtime focussed
webrtcbin, Id like to clocksync audio and video compensating any
possible a/v offset (like e.g. audio being muxed 2 seconds ahead of
video) as well as decoder delays.

Anyways, thats a fine tuning for later times. I get a reasonably
working stream with many sync variants for now..

Interestingly, you are associating "duration/position working" with
"seeking working" to some extend. For my experiments duration/position
works well in any setup, in case I ask decodebin directly.

Seeking however, is still not working. I made a slightly simplified
pipeline, trying to follow your advices, where streaming and
duration/position still works, while seek fails, as in any other
attempt:

------------------------------------------------------------------------------
filesrc location=/home/ubuntu/test.mkv do-timestamp=1 ! decodebin name=decbin
! video/x-raw ! vaapipostproc ! video/x-raw... ! queue  ! vaapih264enc ... !
video/x-h264... ! h264parse config-interval=-1 ! clocksync  !
rtph264pay config-interval=-1 !
application/x-rtp... ! queue ! tee name=videotee ! queue ! fakesink
name=vfakesink

decbin. ! audio/x-raw ! audioconvert ! audioresample ! audio/x-raw... !
opusenc bitrate=... ! queue ! clocksync ! rtpopuspay !
application/x-rtp... ! queue ! tee name=audiotee ! queue ! fakesink
------------------------------------------------------------------------------

webrtcbin consumers are linked to the tees dynamically (with queues).
Seek is sent to the pipeline object.
Any further ideas what I could try here?

Philipp

Am Di., 19. Apr. 2022 um 09:04 Uhr schrieb Marianna Smidth Buschle
<msb at qtec.com>:
>
> On 19.04.2022 08.20, gstreamer-devel-request at lists.freedesktop.org wrote:
>
> Hello mailing list members,
>
> I am still working on a "media (file) player over WebRTC": A file is
> decoded into audio/video and transmitted over webrtcbin.
> As WebRTCBin requires (AFAIK) live input, I use the usual tricks to
> convert multifilesrc into a live source.
>
> My pipeline roughly looks like:
>
> multifilesrc do-timestamp=1 ! (..queues, decoders, encoders) !
> queue ! tee ! queue ! fakesink
>               \- ! queue ! webrtcbin
>
> (not sure if the ASCII art survives the mail encoding, I basically
> have a fakesink as default sink linked to a tee, so that the media is played
> even when no consumers are connected)
>
> I am experimenting with "identity sync=true" at various positions,
> currently I plan to have it in the audio leg only, but all that will be
> finalized later.
>
> You can make your fakesink with sync=true which should give you a similar behavior than to "identity sync=true", at least on that branch of the 'tee'.
>
> Otherwise you should also be able to use 'clocksync' instead of "identity sync=true", which should also do the same.
> This is at least how I handle 'filesrc's.
>
> filesrc name=replay location=/home/msb/test.ts ! decodebin ! identity sync=true ! videoconvert ! ximagesink sync=0
>
> Note that both "identity sync=true" and 'clocksync' only seem to work on demuxed streams (which makes good sense).
>
> My problem is, that I cannot seek while playback happens. Obviously,
> a real live source does not support seek, so its not too surprising.
> Anyways, I like to mimic the behaviour you would get when feeding
> a VCR/DVD player into video in which is grabbed as a live source then.
>
> I have no clue why seek actually fails. I could imagine, its because of
> "identity sync=true" elements, but I tested without them, and its still
> not working. Maybe webrtcbin does not support seeking at all?
>
> In my experience stream duration and seek don't seem to work with multifilesrc.
> They seem to work with filesrc as long as the data is demuxed.
>
> Seek and duration/position queries work (also with sync=0)
>
> filesrc name=replay location=/home/msb/test.ts ! decodebin ! identity sync=true ! videoconvert ! ximagesink sync=1
>
> duration/position queries DON'T work
>
> filesrc name=replay location=/home/msb/test.ts ! fakesink
>
> duration/position queries work
>
> filesrc name=replay location=/home/msb/test.ts ! tsdemux ! fakesink
>
> duration/position queries DON'T work
>
> multifilesrc name=replay location=/home/msb/test.ts ! tsdemux ! fakesink
>
> Seek and duration query DON'T work (position query works, but position increases for each loop)
>
> multifilesrc name=replay location=/home/msb/test.ts ! decodebin ! identity sync=true ! videoconvert ! ximagesink sync=0
>
> If I send the seek to the pipe (or, in most other cases) it returns false.
> (Which is somewhat confusing for me, as the documentation states
> "In a completely prerolled PAUSED or PLAYING pipeline, seeking is always
> guaranteed to return TRUE on a seekable media")
>
> In case I send the seek to the filesrc directly (which I assume to be wrong),
> it returns true, but the pipeline simply stops.
>
> I tried to read about pipeline states, tried diffferent seek modes,
> but I couldnt find anything that would explain my issue or change behaviour.
>
> Any help appreciated! Is there any specific issue or underlying concept
> I need to understand?
>
> Thanks!
> Philipp
>
> --
> Best regards / Med venlig hilsen
> “Marianna Smidth Buschle”