[EXTERNAL]Re: Asking for examples of rsonvif

Thu Jul 18 21:38:51 UTC 2024

Hi Mathieu, thanks for your answers. I'm using the suggested pipeline as a client. I can see the video, but without the overlay (no bounding box).
Using another pipeline to see the metadata XML only, the structure and information seems correct.
--
<?xml version='1.0' encoding='utf-8'?>
<tt:MetadataStream xmlns:tt="http://www.onvif.org/ver10/schema">
    <tt:VideoAnalytics>
        <tt:Frame UtcTime="2024-07-18T18:10:25.554790Z">
            <tt:Object ObjectId="0">
                <tt:Appearance>
                    <tt:Shape>
                        <tt:BoundingBox left="348.268" bottom="55.976" right="401.388" top="114.761"/>
                        <tt:CenterOfGravity x="374.828" y="85.368"/>
                        <tt:Polygon>
                            <tt:Point x="348.268" y="55.976"/>
                            <tt:Point x="401.388" y="55.976"/>
                            <tt:Point x="401.388" y="114.761"/>
                            <tt:Point x="348.268" y="114.761"/>
                        </tt:Polygon>
                    </tt:Shape>
                    <tt:Class>
                        <tt:Type Likelihood="0.508">bus</tt:Type>
                    </tt:Class>
                </tt:Appearance>
            </tt:Object>
        </tt:Frame>
    </tt:VideoAnalytics>
</tt:MetadataStream>
--

But, there are several WARNGING  in the terminal:

--
0:00:14.668628093 2871277 0x783424000e20 WARN      onvifmetadataparse net/onvif/src/onvifmetadataparse/imp.rs:706:gstrsonvif::onvifmetadataparse::imp::OnvifMetadataParse::drain:<onvifmetadataparse0> UTC time 1091757:30:20.516320000 outside segment
0:00:16.528620317 2871277 0x783424000b90 WARN    onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.031219222-1091757:30:22.064552555 yet
0:00:16.561956797 2871277 0x783424000b90 WARN    onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.064552555-1091757:30:22.097885888 yet
0:00:16.595305922 2871277 0x783424000b90 WARN    onvifmetadatacombiner net/onvif/src/onvifmetadatacombiner/imp.rs:303:gstrsonvif::onvifmetadatacombiner::imp::OnvifMetadataCombiner::consume_media:<omc> Timed out but did not receive all meta for media buffer from 1091757:30:22.097885888-1091757:30:22.131219221 yet
--

Using this pipeline to retrieve  metadata only, it seems to work as well:

#  gst-launch-1.0 rtspsrc location="rtsp://localhost:8555/stream" protocols=tcp ! rtponvifmetadatadepay  ! fakesink dump=true
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://localhost:8555/stream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
00000000 (0x739aa002cfdc): 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 27 31  <?xml version='1
00000010 (0x739aa002cfec): 2e 30 27 20 65 6e 63 6f 64 69 6e 67 3d 27 75 74  .0' encoding='ut
00000020 (0x739aa002cffc): 66 2d 38 27 3f 3e 0a 3c 74 74 3a 4d 65 74 61 64  f-8'?>.<tt:Metad
00000030 (0x739aa002d00c): 61 74 61 53 74 72 65 61 6d 20 78 6d 6c 6e 73 3a  ataStream xmlns:
00000040 (0x739aa002d01c): 74 74 3d 22 68 74 74 70 3a 2f 2f 77 77 77 2e 6f  tt="http://www.o
00000050 (0x739aa002d02c): 6e 76 69 66 2e 6f 72 67 2f 76 65 72 31 30 2f 73  nvif.org/ver10/s
00000060 (0x739aa002d03c): 63 68 65 6d 61 22 3e 3c 74 74 3a 56 69 64 65 6f  chema"><tt:Video
00000070 (0x739aa002d04c): 41 6e 61 6c 79 74 69 63 73 3e 3c 74 74 3a 46 72  Analytics><tt:Fr
00000080 (0x739aa002d05c): 61 6d 65 20 55 74 63 54 69 6d 65 3d 22 32 30 32  ame UtcTime="202
00000090 (0x739aa002d06c): 34 2d 30 37 2d 31 38 54 32 31 3a 33 34 3a 31 35  4-07-18T21:34:15
000000a0 (0x739aa002d07c): 2e 36 32 32 38 36 36 5a 22 3e 3c 74 74 3a 4f 62  .622866Z"><tt:Ob
000000b0 (0x739aa002d08c): 6a 65 63 74 20 4f 62 6a 65 63 74 49 64 3d 22 30  ject ObjectId="0
000000c0 (0x739aa002d09c): 22 3e 3c 74 74 3a 41 70 70 65 61 72 61 6e 63 65  "><tt:Appearance
000000d0 (0x739aa002d0ac): 3e 3c 74 74 3a 53 68 61 70 65 3e 3c 74 74 3a 42  ><tt:Shape><tt:B
000000e0 (0x739aa002d0bc): 6f 75 6e 64 69 6e 67 42 6f 78 20 6c 65 66 74 3d  oundingBox left=
000000f0 (0x739aa002d0cc): 22 33 39 30 2e 30 31 30 22 20 62 6f 74 74 6f 6d  "390.010" bottom
00000100 (0x739aa002d0dc): 3d 22 31 38 32 2e 30 35 39 22 20 72 69 67 68 74  ="182.059" right
00000110 (0x739aa002d0ec): 3d 22 34 35 31 2e 36 35 33 22 20 74 6f 70 3d 22  ="451.653" top="
00000120 (0x739aa002d0fc): 32 32 39 2e 30 35 30 22 20 2f 3e 3c 74 74 3a 43  229.050" /><tt:C
00000130 (0x739aa002d10c): 65 6e 74 65 72 4f 66 47 72 61 76 69 74 79 20 78  enterOfGravity x
00000140 (0x739aa002d11c): 3d 22 34 32 30 2e 38 33 32 22 20 79 3d 22 32 30  ="420.832" y="20
00000150 (0x739aa002d12c): 35 2e 35 35 35 22 20 2f 3e 3c 74 74 3a 50 6f 6c  5.555" /><tt:Pol
00000160 (0x739aa002d13c): 79 67 6f 6e 3e 3c 74 74 3a 50 6f 69 6e 74 20 78  ygon><tt:Point x
00000170 (0x739aa002d14c): 3d 22 33 39 30 2e 30 31 30 22 20 79 3d 22 31 38  ="390.010" y="18
00000180 (0x739aa002d15c): 32 2e 30 35 39 22 20 2f 3e 3c 74 74 3a 50 6f 69  2.059" /><tt:Poi
00000190 (0x739aa002d16c): 6e 74 20 78 3d 22 34 35 31 2e 36 35 33 22 20 79  nt x="451.653" y
000001a0 (0x739aa002d17c): 3d 22 31 38 32 2e 30 35 39 22 20 2f 3e 3c 74 74  ="182.059" /><tt
000001b0 (0x739aa002d18c): 3a 50 6f 69 6e 74 20 78 3d 22 34 35 31 2e 36 35  :Point x="451.65
000001c0 (0x739aa002d19c): 33 22 20 79 3d 22 32 32 39 2e 30 35 30 22 20 2f  3" y="229.050" /
000001d0 (0x739aa002d1ac): 3e 3c 74 74 3a 50 6f 69 6e 74 20 78 3d 22 33 39  ><tt:Point x="39
000001e0 (0x739aa002d1bc): 30 2e 30 31 30 22 20 79 3d 22 32 32 39 2e 30 35  0.010" y="229.05
000001f0 (0x739aa002d1cc): 30 22 20 2f 3e 3c 2f 74 74 3a 50 6f 6c 79 67 6f  0" /></tt:Polygo
00000200 (0x739aa002d1dc): 6e 3e 3c 2f 74 74 3a 53 68 61 70 65 3e 3c 74 74  n></tt:Shape><tt
00000210 (0x739aa002d1ec): 3a 43 6c 61 73 73 3e 3c 74 74 3a 54 79 70 65 20  :Class><tt:Type
00000220 (0x739aa002d1fc): 4c 69 6b 65 6c 69 68 6f 6f 64 3d 22 30 2e 33 30  Likelihood="0.30
00000230 (0x739aa002d20c): 35 22 3e 62 65 6e 63 68 3c 2f 74 74 3a 54 79 70  5">bench</tt:Typ
00000240 (0x739aa002d21c): 65 3e 3c 2f 74 74 3a 43 6c 61 73 73 3e 3c 2f 74  e></tt:Class></t
00000250 (0x739aa002d22c): 74 3a 41 70 70 65 61 72 61 6e 63 65 3e 3c 2f 74  t:Appearance></t
00000260 (0x739aa002d23c): 74 3a 4f 62 6a 65 63 74 3e 3c 2f 74 74 3a 46 72  t:Object></tt:Fr
00000270 (0x739aa002d24c): 61 6d 65 3e 3c 2f 74 74 3a 56 69 64 65 6f 41 6e  ame></tt:VideoAn
00000280 (0x739aa002d25c): 61 6c 79 74 69 63 73 3e 3c 2f 74 74 3a 4d 65 74  alytics></tt:Met
00000290 (0x739aa002d26c): 61 64 61 74 61 53 74 72 65 61 6d 3e              adataStream>

Any suggestions to try?
Thanks!

Matias

________________________________
De: Mathieu Duponchelle <mathieu at centricular.com>
Enviado: viernes, 28 de junio de 2024 12:59
Para: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Cc: Matias Varela <matias.varela at epicio.com>
Asunto: [EXTERNAL]Re: Asking for examples of rsonvif

Hello Matias,

Yes, documentation is severely lacking for those elements :| In practice on the receiving side I was running pipelines such as:

gst-launch-1.0 onvifmetadatacombiner latency=500000000 name=omc ! videoconvert ! onvifmetadataoverlay ! videoconvert ! autovideosink rtspsrc latency=500 add-reference-timestamp-meta=true location="redacted" name=src src. ! application/x-rtp, encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! omc.media<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fomc.media&c=E,1,4zAiTwCr487ra1LUNQ5U1U0L6rCtRUUhTjiWlfaWHocLRLCrMFpwuWrhZsm5jsmtbV5F3XmC6za01BUiZmJN2A9q0b1C8EXgT0oVnu_cfcYK&typo=1&ancr_add=1> src. ! rtponvifmetadatadepay ! onvifmetadataparse ! omc.meta

On the sender side, you are going to want to produce XML data according to the ONVIF specification, route it through rtponvifmetadatapay and send it over, I have not done any sort of authoring as part of my work on this to be honest :)

Wishing you luck!

On Thu, 2024-06-27 at 21:19 +0000, Matias Varela via gstreamer-devel wrote:

Hi everyone.

I'm very new to gstreamer and also to this email list, but I can't find any information on how to use rsonvif (https://gstreamer.freedesktop.org/documentation/rsonvif<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fgstreamer.freedesktop.org%2fdocumentation%2frsonvif&c=E,1,pH3nXq7ETm7txKXbelu4m3Ippj7ELfS5YZ6m8CJ9FgisJaWFZPyVj4mNxC5z4BuaB1Sg36DJFUGRYCzbLg8S8ju_w2oOQSzBkBjxqLYOTbCirTAyHg,,&typo=1>)

I'm building an application to take a rtsp stream, detect and track some objects, and then place this metadata according to the ONVIF Profile M specification (using rtponvifmetadatapay). In the end, I need to use onvifmetadataoverlay to see both things together in the player.
This is the general idea, so any information to point me in the right direction is appreciated.

I saw the code https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/onvif/src?ref_type=heads,<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fgitlab.freedesktop.org%2fgstreamer%2fgst-plugins-rs%2f-%2ftree%2fmain%2fnet%2fonvif%2fsrc%3fref_type%3dheads%2c&c=E,1,qi3tpwNSbByD3j50cN_HOMRTN0PXGLydCRwgk4nMlaclynmN1ED14iBckLN2Oqh-XZxa2UTmi8O2rhyJshoYtHTM2Micmk0AGTcs1gZWHrXxMUlp2_093Q,,&typo=1&ancr_add=1> and, beyond a relative new with rust, the code do not have comments or readme to know how I can use it.

Thanks!

matuu

-- Matias Varela | Backend Engineer

AI | IoT | Connectivity
www.epicio.com<https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.epicio.com%2f&c=E,1,jTfcXbf224yu19YXegPrsf7GN9O6jZJURIxAGkafsrhgFUJqKPoYadokVhvcuiq2o6dmrpaPBOgrqN-pctOq6GWw4MpyIOw9a0cufWX5tg,,&typo=1>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20240718/30f342bf/attachment-0001.htm>