[EXTERNAL]Re: Asking for examples of rsonvif
Mathieu Duponchelle
mathieu at centricular.com
Mon Jul 22 17:47:04 UTC 2024
Hey,
Getting metadata is good news, but only half the battle if you want to pair it with the relevant video frames: the UtcTime as parsed in the metadata needs to match with the reference timestamp meta on the video frame.
I'm not sure what the exact specifics of your issue are here, but I would recommend looking at the debug / trace logs from onvifmetadataparse to understand what's happening better, the warnings you shared are indeed probably causing the lack of bounding boxes displayed :)
Cheers,
Mathieu
On Thu, 2024-07-18 at 21:38 +0000, Matias Varela wrote:
> Hi Mathieu, thanks for your answers. I'm using the suggested pipeline as a client. I can see the video, but without the overlay (no bounding box).
> Using another pipeline to see the metadata XML only, the structure and information seems correct.
> --
> <?xml version='1.0' encoding='utf-8'?>
> <tt:MetadataStream xmlns:tt="http://www.onvif.org/ver10/schema">
> <tt:VideoAnalytics>
> <tt:Frame UtcTime="2024-07-18T18:10:25.554790Z">
> <tt:Object ObjectId="0">
> <tt:Appearance>
> <tt:Shape>
> <tt:BoundingBox left="348.268" bottom="55.976" right="401.388" top="114.761"/>
> <tt:CenterOfGravity x="374.828" y="85.368"/>
> <tt:Polygon>
> <tt:Point x="348.268" y="55.976"/>
> <tt:Point x="401.388" y="55.976"/>
> <tt:Point x="401.388" y="114.761"/>
> <tt:Point x="348.268" y="114.761"/>
> </tt:Polygon>
> </tt:Shape>
> <tt:Class>
> <tt:Type Likelihood="0.508">bus</tt:Type>
> </tt:Class>
> </tt:Appearance>
> </tt:Object>
> </tt:Frame>
> </tt:VideoAnalytics>
> </tt:MetadataStream>
> --
>
>
> But, there are several WARNGING in the terminal:
>
>
> --
> 0:00:14.668628093 2871277 0x783424000e20 WARN onvifmetadataparse net/onvif/src/onvifmetadataparse/imp.rs:706:gstrsonvif::onvifmetadataparse::imp::OnvifMetadataParse::drain:<onvifmetadataparse0> UTC time 1091757:30:20.516320000 outside segment
> 0:00:16.528620317 2871277 0x783424000b90 WARN onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.031219222-1091757:30:22.064552555 yet
> 0:00:16.561956797 2871277 0x783424000b90 WARN onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.064552555-1091757:30:22.097885888 yet
> 0:00:16.595305922 2871277 0x783424000b90 WARN onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.097885888-1091757:30:22.131219221 yet
> --
>
>
>
>
> Using this pipeline to retrieve metadata only, it seems to work as well:
>
>
> # gst-launch-1.0 rtspsrc location="rtsp://localhost:8555/stream" protocols=tcp ! rtponvifmetadatadepay ! fakesink dump=true
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Progress: (open) Opening Stream
> Pipeline is PREROLLED ...
> Prerolled, waiting for progress to finish...
> Progress: (connect) Connecting to rtsp://localhost:8555/stream
> Progress: (open) Retrieving server options
> Progress: (open) Retrieving media info
> Progress: (request) SETUP stream 0
> Progress: (request) SETUP stream 1
> Progress: (open) Opened Stream
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Progress: (request) Sending PLAY request
> Progress: (request) Sending PLAY request
> Progress: (request) Sent PLAY request
> Redistribute latency...
> Redistribute latency...
> Redistribute latency...
> 00000000 (0x739aa002cfdc): 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 27 31 <?xml version='1
> 00000010 (0x739aa002cfec): 2e 30 27 20 65 6e 63 6f 64 69 6e 67 3d 27 75 74 .0' encoding='ut
> 00000020 (0x739aa002cffc): 66 2d 38 27 3f 3e 0a 3c 74 74 3a 4d 65 74 61 64 f-8'?>.<tt:Metad
> 00000030 (0x739aa002d00c): 61 74 61 53 74 72 65 61 6d 20 78 6d 6c 6e 73 3a ataStream xmlns:
> 00000040 (0x739aa002d01c): 74 74 3d 22 68 74 74 70 3a 2f 2f 77 77 77 2e 6f tt="http://www.o
> 00000050 (0x739aa002d02c): 6e 76 69 66 2e 6f 72 67 2f 76 65 72 31 30 2f 73 nvif.org/ver10/s
> 00000060 (0x739aa002d03c): 63 68 65 6d 61 22 3e 3c 74 74 3a 56 69 64 65 6f chema"><tt:Video
> 00000070 (0x739aa002d04c): 41 6e 61 6c 79 74 69 63 73 3e 3c 74 74 3a 46 72 Analytics><tt:Fr
> 00000080 (0x739aa002d05c): 61 6d 65 20 55 74 63 54 69 6d 65 3d 22 32 30 32 ame UtcTime="202
> 00000090 (0x739aa002d06c): 34 2d 30 37 2d 31 38 54 32 31 3a 33 34 3a 31 35 4-07-18T21:34:15
> 000000a0 (0x739aa002d07c): 2e 36 32 32 38 36 36 5a 22 3e 3c 74 74 3a 4f 62 .622866Z"><tt:Ob
> 000000b0 (0x739aa002d08c): 6a 65 63 74 20 4f 62 6a 65 63 74 49 64 3d 22 30 ject ObjectId="0
> 000000c0 (0x739aa002d09c): 22 3e 3c 74 74 3a 41 70 70 65 61 72 61 6e 63 65 "><tt:Appearance
> 000000d0 (0x739aa002d0ac): 3e 3c 74 74 3a 53 68 61 70 65 3e 3c 74 74 3a 42 ><tt:Shape><tt:B
> 000000e0 (0x739aa002d0bc): 6f 75 6e 64 69 6e 67 42 6f 78 20 6c 65 66 74 3d oundingBox left=
> 000000f0 (0x739aa002d0cc): 22 33 39 30 2e 30 31 30 22 20 62 6f 74 74 6f 6d "390.010" bottom
> 00000100 (0x739aa002d0dc): 3d 22 31 38 32 2e 30 35 39 22 20 72 69 67 68 74 ="182.059" right
> 00000110 (0x739aa002d0ec): 3d 22 34 35 31 2e 36 35 33 22 20 74 6f 70 3d 22 ="451.653" top="
> 00000120 (0x739aa002d0fc): 32 32 39 2e 30 35 30 22 20 2f 3e 3c 74 74 3a 43 229.050" /><tt:C
> 00000130 (0x739aa002d10c): 65 6e 74 65 72 4f 66 47 72 61 76 69 74 79 20 78 enterOfGravity x
> 00000140 (0x739aa002d11c): 3d 22 34 32 30 2e 38 33 32 22 20 79 3d 22 32 30 ="420.832" y="20
> 00000150 (0x739aa002d12c): 35 2e 35 35 35 22 20 2f 3e 3c 74 74 3a 50 6f 6c 5.555" /><tt:Pol
> 00000160 (0x739aa002d13c): 79 67 6f 6e 3e 3c 74 74 3a 50 6f 69 6e 74 20 78 ygon><tt:Point x
> 00000170 (0x739aa002d14c): 3d 22 33 39 30 2e 30 31 30 22 20 79 3d 22 31 38 ="390.010" y="18
> 00000180 (0x739aa002d15c): 32 2e 30 35 39 22 20 2f 3e 3c 74 74 3a 50 6f 69 2.059" /><tt:Poi
> 00000190 (0x739aa002d16c): 6e 74 20 78 3d 22 34 35 31 2e 36 35 33 22 20 79 nt x="451.653" y
> 000001a0 (0x739aa002d17c): 3d 22 31 38 32 2e 30 35 39 22 20 2f 3e 3c 74 74 ="182.059" /><tt
> 000001b0 (0x739aa002d18c): 3a 50 6f 69 6e 74 20 78 3d 22 34 35 31 2e 36 35 :Point x="451.65
> 000001c0 (0x739aa002d19c): 33 22 20 79 3d 22 32 32 39 2e 30 35 30 22 20 2f 3" y="229.050" /
> 000001d0 (0x739aa002d1ac): 3e 3c 74 74 3a 50 6f 69 6e 74 20 78 3d 22 33 39 ><tt:Point x="39
> 000001e0 (0x739aa002d1bc): 30 2e 30 31 30 22 20 79 3d 22 32 32 39 2e 30 35 0.010" y="229.05
> 000001f0 (0x739aa002d1cc): 30 22 20 2f 3e 3c 2f 74 74 3a 50 6f 6c 79 67 6f 0" /></tt:Polygo
> 00000200 (0x739aa002d1dc): 6e 3e 3c 2f 74 74 3a 53 68 61 70 65 3e 3c 74 74 n></tt:Shape><tt
> 00000210 (0x739aa002d1ec): 3a 43 6c 61 73 73 3e 3c 74 74 3a 54 79 70 65 20 :Class><tt:Type
> 00000220 (0x739aa002d1fc): 4c 69 6b 65 6c 69 68 6f 6f 64 3d 22 30 2e 33 30 Likelihood="0.30
> 00000230 (0x739aa002d20c): 35 22 3e 62 65 6e 63 68 3c 2f 74 74 3a 54 79 70 5">bench</tt:Typ
> 00000240 (0x739aa002d21c): 65 3e 3c 2f 74 74 3a 43 6c 61 73 73 3e 3c 2f 74 e></tt:Class></t
> 00000250 (0x739aa002d22c): 74 3a 41 70 70 65 61 72 61 6e 63 65 3e 3c 2f 74 t:Appearance></t
> 00000260 (0x739aa002d23c): 74 3a 4f 62 6a 65 63 74 3e 3c 2f 74 74 3a 46 72 t:Object></tt:Fr
> 00000270 (0x739aa002d24c): 61 6d 65 3e 3c 2f 74 74 3a 56 69 64 65 6f 41 6e ame></tt:VideoAn
> 00000280 (0x739aa002d25c): 61 6c 79 74 69 63 73 3e 3c 2f 74 74 3a 4d 65 74 alytics></tt:Met
> 00000290 (0x739aa002d26c): 61 64 61 74 61 53 74 72 65 61 6d 3e adataStream>
>
>
> Any suggestions to try?
> Thanks!
>
>
> Matias
>
>
> **De:** Mathieu Duponchelle <mathieu at centricular.com>
> **Enviado:** viernes, 28 de junio de 2024 12:59
> **Para:** Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
> **Cc:** Matias Varela <matias.varela at epicio.com>
> **Asunto:** [EXTERNAL]Re: Asking for examples of rsonvif
>
> Hello Matias,
> Yes, documentation is severely lacking for those elements :| In practice on the receiving side I was running pipelines such as:
> ```
> gst-launch-1.0 onvifmetadatacombiner latency=500000000 name=omc ! videoconvert ! onvifmetadataoverlay ! videoconvert ! autovideosink rtspsrc latency=500 add-reference-timestamp-meta=true location="redacted" name=src src. ! application/x-rtp, encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! [omc.media](https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fomc.media&c=E,1,4zAiTwCr487ra1LUNQ5U1U0L6rCtRUUhTjiWlfaWHocLRLCrMFpwuWrhZsm5jsmtbV5F3XmC6za01BUiZmJN2A9q0b1C8EXgT0oVnu_cfcYK&typo=1&ancr_add=1) src. ! rtponvifmetadatadepay ! onvifmetadataparse ! omc.meta
>
>
> ```
> On the sender side, you are going to want to produce XML data according to the ONVIF specification, route it through `rtponvifmetadatapay` and send it over, I have not done any sort of authoring as part of my work on this to be honest :)
> Wishing you luck!
> On Thu, 2024-06-27 at 21:19 +0000, Matias Varela via gstreamer-devel wrote:
>
> > Hi everyone.
> > I'm very new to gstreamer and also to this email list, but I can't find any information on how to use rsonvif ([https://gstreamer.freedesktop.org/documentation/rsonvif](https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fgstreamer.freedesktop.org%2fdocumentation%2frsonvif&c=E,1,pH3nXq7ETm7txKXbelu4m3Ippj7ELfS5YZ6m8CJ9FgisJaWFZPyVj4mNxC5z4BuaB1Sg36DJFUGRYCzbLg8S8ju_w2oOQSzBkBjxqLYOTbCirTAyHg,,&typo=1))
> > I'm building an application to take a rtsp stream, detect and track some objects, and then place this metadata according to the ONVIF Profile M specification (using rtponvifmetadatapay). In the end, I need to use onvifmetadataoverlay to see both things together in the player.
> > This is the general idea, so any information to point me in the right direction is appreciated.
> > I saw the code [
> > https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/onvif/src?ref_type=heads,](https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fgitlab.freedesktop.org%2fgstreamer%2fgst-plugins-rs%2f-%2ftree%2fmain%2fnet%2fonvif%2fsrc%3fref_type%3dheads%2c&c=E,1,qi3tpwNSbByD3j50cN_HOMRTN0PXGLydCRwgk4nMlaclynmN1ED14iBckLN2Oqh-XZxa2UTmi8O2rhyJshoYtHTM2Micmk0AGTcs1gZWHrXxMUlp2_093Q,,&typo=1&ancr_add=1) and, beyond a relative new with rust, the code do not have comments or readme to know how I can use it.
> > Thanks!
> > matuu
> > -- **Matias Varela** | Backend Engineer
> > **AI | IoT | Connectivity**
> > [www.epicio.com](https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.epicio.com%2f&c=E,1,jTfcXbf224yu19YXegPrsf7GN9O6jZJURIxAGkafsrhgFUJqKPoYadokVhvcuiq2o6dmrpaPBOgrqN-pctOq6GWw4MpyIOw9a0cufWX5tg,,&typo=1)
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20240722/436e6e52/attachment-0001.htm>
More information about the gstreamer-devel
mailing list