Problem of sync between multiple pipeline and meaning of PTS and DTS

[이병헌]

Thanks a lot!
I'm gonna do more investigation with your advice.


2021년 8월 27일 (금) 오전 5:03, Nicolas Dufresne <nicolas at ndufresne.ca>님이 작성:

> Le vendredi 27 août 2021 à 03:32 +0900, 이병헌 a écrit :
> > Thanks for your wonderful answer
> >
> > Now I can understand about qurstion #2 and I will do more investigation
> on #3 as
> > you said.
> > But I still can't understand concept of running-time and segment you
> said.
> >
> > Before all, I would explain my problem again.
> > I need common absolute clock between many rtsp cameras to compare
> consistence of
> > each buffers from those cameras.
> > I will compare those buffers to know which buffers are recored at the
> same time.
> > This mean I need to get absolute recoded time from rtsp packets.
> > So I wanna know whether it possible or not
> > And If possible then how
> >
> > First, after I get your answer, I tried to read gstreamer document about
> segment
> > and running time.
> > In my understand, segment is a bundle of data that composed of buffers.
> > And buffer running time is relative timescale between buffers to play on
> right
> > time.
> > And segment event is event to set segment start and rate and some more
> of stream
> > inform
> >
> > So I have more questions here
> >
> > 1. can you explain more about segment, segment event and running time
> with
> > specific example?
>
>
> https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/clock.html?gi-language=c#clock-runningtime
>
> As you are targeting RTP, you might need to learn how to enable RFC7273 to
> get
> more accurate reconstruction of the remote clock and timestamps (I don't
> myself
> know how to use this). But as soon as you got this right, there should be
> no
> difference, and running-time becomes the canonical timescale to compare
> timestamp against each other.
>
>
> > 2. If I don't use any segment event while stream, then PTS of buffer is
> always
> > same as running time?
>
> This is quite possible that your segment are start=0 without any offset,
> but I
> would not rely on this, as not using a segment in one element/decoder
> other is
> no an API. If we have a good reason to modify the segment later on, your
> application will break and it will be application fault.
>
> >
> > Sorry for being dumb..
> >
> > Thanks
> >
> >
> >
> >
> >
> >
> > 2021년 8월 26일 (목) 오후 10:46, Nicolas Dufresne <nicolas at ndufresne.ca>님이 작성:
> > > Le jeudi 26 août 2021 à 18:45 +0900, 이병헌 via gstreamer-devel a écrit :
> > > > Hi, I’m new on gstreamer now
> > > > And my first language is not an English
> > > > So if my questions are not good to be understood I would say sorry
> first..
> > > >
> > > > Anyway now I’m trying to make application that need function of
> > > > synchronize
> > > > buffers between multiple pipeline
> > > > So I’m trying to sync those buffers by Lockstep method with their
> > > > timestamp
> > > > value in those buffers
> > > > But there have three problem
> > > >
> > > > This is my pipeline blueprint
> > > >
> > > > rtspsrc ! h265depay ! av_dech265 ! videoscale ! videoconvert !
> caps-filter
> > > > !
> > > > queue ! videoscale ! tensor_converter ! tensor_filter ! tensor_sink
> —— get
> > > > buffer from sink by callback and put buffers to — > Ring buffer - - \
> > > > ! rtspsrc ! h265depay ! av_dech265 ! videoscale ! videoconvert !
> caps-
> > > > filter
> > > !
> > > > queue ! videoscale ! tensor_converter ! tensor_filter ! tensor_sink
> —— get
> > > > buffer from sink by callback and put buffers to — > Ring buffer —- -
> >
> > > > lockstep sync —> some association logics ...
> > > > ! rtspsrc ! h265depayl ! av_dech265 ! videoscale ! videoconvert !
> caps-
> > > filter
> > > > ! queue ! videoscale ! tensor_converter ! tensor_filter !
> tensor_sink ——
> > > > get
> > > > buffer from sink by callback and put buffers to — > Ring buffer —/
> > > >
> > > > First , PTS is not wall clock time or ntp time
> > > > Second, DTS value is always -1
> > > > Third, when I check pts values at source element, pts values are
> always
> > > > increment. But when I check my ring buffer element, some value of
> pts are
> > > > decrease between adjacent buffer, this mean order of input buffer
> can be
> > > > shuffled while streaming
> > > >
> > > > So.. my questions are these
> > > >
> > > > 1. How to get wall clock PTS value from rtsp packets in gstreamer?
> > >
> > > In GStreamer we have the concept of "running-time", this is the
> timescale
> > > you
> > > must use to compare GST_BUFFER_DTS_OR_PTS() (that macro takes care of
> > > picking
> > > the right TS) against each other.
> > >
> > > In order to get running-time from a buffer PTS, you have to track the
> > > GstSegment
> > > that is propagated using the GST_EVENT_SEGMENT. You can then convert
> using
> > > gst_segment_to_running_time() or gst_segment_to_running_time_full().
> The
> > > full
> > > version supports negative running time, this might not be needed for
> your
> > > use
> > > case.
> > >
> > > > 2. Meaning of DTS value -1
> > >
> > > DTS stands for decoting timestamp. In your example, you are dealing
> with raw
> > > data, so decoding does not make sense, so that TS is set to
> > > GST_CLOCK_TIME_INVALID. It's not really -1, since this is unsigned
> integer.
> > >
> > > Note that DTS is optional, it is mostly useful for streams were the
> decoding
> > > order differ from the presentation order, or when there is a delay
> (like in
> > > opus) with some initial data that should be skipped.
> > >
> > > > 3. Is it possible to get different order of buffer from input buffer
> order
> > > at
> > > > some elements in pipeline? If can, then do I have to sort those
> buffers
> > > > with
> > > > their timestamp with some priority queue rather than ring buffer ?
> > >
> > > In your case, it looks like getting backward timestamp is a bug. Could
> be on
> > > the
> > > RTSP server side, this requires deep investigation to figure-out.
> > >
> > > >
> > > > Sorry for writing that hard to understand
> > > > and I would give more inform if it is not enough to answer
> > > >
> > > > Thanks
> > >
> > >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210827/3c54d06b/attachment-0001.htm>