Get camera timestamp from rtsp stream.

[Anand Sivaram]

I think Jeff was talking about this one.

https://www.onvif.org/specs/stream/ONVIF-Streaming-Spec.pdf
6.3 section RTP header extension
That would require the camera to support it.

@Anton - got your requirement.  If the camera does not support any extra
timing as described above, then I am not sure how to get the exact actual
capture timestamp of the frame from the camera.

We can try to derive it based on RTP timestamp only.
- With splitmuxsink and "format-location" callback, get the system time -
for example time.time() equivalent of python - on the RTSP client and mark
it in the MP4 filename.
- RTP clock gets incremented by 90000Hz and if it is 30fps then 3000 in
every frame.
- Since you are using 3sec MP4, then there would be 90 frames, 270000 in
RTP timestamp increase.
- Now run the recording for some time, correlate the actual time observed
on "format-location" and the RTP timestamp and compute the jitter and see
if it is sufficient.

This method has one assumption - that is the network jitter is low and
manageable..  If the camera and RTSP client are on the same network -
preferably wired - then jitter would be a few msec only, then that could be
less than 10% of interframe time of 33msec.

Anyway, would be happy to hear better suggestions from experts.

Thanks and Regards

Anand

On Wed, 19 Jul 2023 at 22:56, Антон Шаров <sharov_am at mail.ru> wrote:

> Hi, Anand!
>
> Thank you so much for your reply!
>
> The thing is I don’t need RTP, I need exact camera(server) capture time of
> a frame.
> Ideally, I need capture time for each frame, but probably for some
> keyframe of mp4 file
> (first frame of this video file?) would be ok also. I’m trying to use my
> custom appsink,
> smth like
> appSink.NewSample += AppSink_NewSample;
>
> splitmuxsink["sink"] = appSink;
> splitmuxsink["max-size-time"] = 3000000000;
> splitmuxsink["async-finalize"] = false;
>
> private static void AppSink_NewSample(object o, NewSampleArgs args)
>         {
>             if (o is AppSink aps)
>             {
>                 var sample = aps.PullSample();
>                 var buf = sample.Buffer;
>                 buf.Map(out var info, MapFlags.Read);
>                 var ts = buf.GetReferenceTimestampMeta();// !!!!!!
>                 buf.Unmap(info);
>             }
>         }
> I don’t quite understand whether it is possible to have needed (3 sec. in
> this case) mp4 file in
> whole buffer, for which I can use GetReferenceTimestampMeta() which return
> me timestamp
> for this buffer (hence for  whole file, hence for keyframe). But on
> practice with my custom sink
> I got some weired chunks in buffer and  GetReferenceTimestampMeta returns
> null (not null exactly,
> but some useless info where needed timestamp = 0).
>
> I believe this ideal approach won’t work (because appsink is not
> seekable), but at least it looks like
> desired solution.
>
> In case of default sink (filesink), I need
>
>    1. change file name, which somehow seems to be possible but is buggy
>    on .net library, but at least it is seems to possible;
>    2. get timestamp of the first frames (buffer) of newly created file.
>    Ideally I would like to name newly created file with timestamp_value.mp4.
>
>
>
>
>
> Среда, 19 июля 2023, 12:37 +03:00 от Anand Sivaram <aspnair at gmail.com>:
>
> Hello Anton,
>
> The RTSP media is coming as separate RTP streams for each video and
> audio.  They must be having only usual RTP parameters like timestamp,
> sequence number and payload type.
> That too RTP timestamps are initialized randomly as per standard, so the
> timestamps on video and audio streams have no relation at all.
> If you are planning to read the time using some OCR software from the
> video frame, then you will have to decode H.264 and the typical granularity
> is 1sec.
>
> Are you using the "format-location" callback signal in splitmuxsink with
> which you can generate custom filenames with timestamp and any prefix.  It
> is not available on gst-launch, but it is definitely available on C and
> Python.
>
> Thanks and Regards
>
> Anand
>
>
> On Mon, 17 Jul 2023 at 15:36, Антон Шаров via gstreamer-devel <
> gstreamer-devel at lists.freedesktop.org
> <//e.mail.ru/compose/?mailto=mailto%3agstreamer%2ddevel at lists.freedesktop.org>>
> wrote:
>
>
> Hi.
>
> I’m given rtsp stream from camera, where each frame has capturing
> timestamp.
> Connection strin looks like rtsp://{usr}:{pwd{@ip_addr
> /onvif/media?profile=Profile1
> I need to store this data for some time and provide search access for this
> data, either return closest (exact) frame for provided timestamp or
> return mp4 file which containts this closest frame. My first pipeline is  rtsp
> ! h264depay ! h264parse ! splitmuxsink  location=… max-time-size=10seconds
> (for example), when I save new file via splitmuxsink, I’m some how need to
> get camera timestamp
> for first frame of video (or key frame) and maybe store this mp4 file as
> timestamp.mp4 (or save ts for later in some db, for example).
>
> Other approach is to use jpegenc and to store each frame with it’s
> timestamp, but I don’t know how to obtain timestamp for jpeg buffer (rtsp !
> decodebin ! jpegenc ! appsink). But  I assume  that this won’t be effective
> solution in terms of CPU and storage usage, and better to store mp4 files.
>
> So, in both cases I don’t know how to get reference-timestamp-meta for
> needed buffer.
>
> Can someone help me?
>
> PS: I use C# wrapper, namely gstreamer-sharp, but I don’t think it is
> relevant for this problem.
>
> Thanks in advance.
>
> --
> С Уважением,
> Шаров Антон
>
>
>
> --
> С Уважением,
> Шаров Антон
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230719/291eb7b2/attachment.htm>