Reduce latency on Win10 for AWS Kinesis Video

[Angela Yan]

Hi Nirbheek and all,

Forgive me to ask a silly question, how should I find out the latency of the pipeline on Win10?

I have tried GST_DEBUG, or GST_TRACERS as I gathered from the web. If I understand it correctly, these 2 are environment variables, so I can't directly run them like what people do in linux:
GST_DEBUG="GST_TRACER:7" GST_TRACERS=latency gst-launch-1.0 ....

So I set them as environment variables but still I don't see any trace output displayed:
C:\Users\angela>set GST_TRACERS="latency"

C:\Users\angela>set GST_DEBUG="GST_TRACER:7"

C:\Users\angela>gst-launch-1.0 mfvideosrc do-timestamp=TRUE device-name="Studio Camera" ! video/x-raw,width=1920,height=1080 ! queue ! videoconvert ! autovideosink
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...

(gst-launch-1.0:16660): GStreamer-CRITICAL **: 15:35:26.989: gst_pad_query_caps: assertion 'GST_IS_PAD (pad)' failed

(gst-launch-1.0:16660): GStreamer-CRITICAL **: 15:35:26.995: gst_object_unref: assertion 'object != NULL' failed

(gst-launch-1.0:16660): GStreamer-CRITICAL **: 15:35:26.999: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:16660): GStreamer-CRITICAL **: 15:35:27.012: gst_mini_object_unref: assertion 'mini_object != NULL' failed
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.d3d11.device.handle=context, device=(GstD3D11Device)"\(GstD3D11Device\)\ d3d11device5", adapter=(uint)0, adapter-luid=(gint64)67806, device-id=(uint)22813, vendor-id=(uint)32902, hardware=(boolean)true, description=(string)"Intel\(R\)\ HD\ Graphics\ P630";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: Output window was closed
Additional debug info:
../sys/d3d11/gstd3d11videosink.cpp(1266): gst_d3d11_video_sink_show_frame (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video
ERROR: from element /GstPipeline:pipeline0/GstMFVideoSrc:mfvideosrc0: Internal data stream error.
Execution ended after 0:03:18.337586000
Setting pipeline to NULL ...
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstMFVideoSrc:mfvideosrc0:
streaming stopped, reason error (-5)
Freeing pipeline ...

Even if I set the GST_DEBUG_FILE environment variable, I can see the file is generated, but there is no content inside.

Can anyone advise how to see the latency of the pipeline in Win10?

Thank you,
Angela
________________________________
From: Nirbheek Chauhan <nirbheek.chauhan at gmail.com>
Sent: Friday, June 24, 2022 3:59 AM
To: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Cc: Angela Yan <yanyan9 at hotmail.com>
Subject: Re: Reduce latency on Win10 for AWS Kinesis Video

Hi Angela,

The ksvideosrc plugin uses the deprecated WinKS API. Try using
mfvideosrc, that should perform better.

You should also add some queues in your pipeline. Also note that you
can use hardware accelerated encoding with the mfh264enc element. Both
these elements belong to the mediafoundation plugin.

Technically there's no reason for there to be 10s latency in capture +
encode. It should be just a few hundred milliseconds. I would
recommend measuring where the latency actually is, for example, what
is the latency in this pipeline?

mfvideosrc ! image/jpeg,width=1920,height=1080 ! jpegdec ! queue !
videoconvert ! autovideosink

Generally you will get JPEG from a USB camera, not YUV. If that's not the case:

mfvideosrc ! video/x-raw,width=1920,height=1080 ! queue ! videoconvert
! autovideosink

Cheers,
Nirbheek

On Thu, Jun 23, 2022 at 2:45 PM Angela Yan via gstreamer-devel
<gstreamer-devel at lists.freedesktop.org> wrote:
>
> Hi all,
>
> I am running gstreamer on Win 10 to stream from a high res camera to AWS Kinesis Video stream through the AWS producer plugin (KVSSink). However, we found significant delay in the whole process. Around ~10 sec delay, with 60% on gstreamer to AWS and 40% on AWS to player.
>
> I have tried added tune=zerolatency on the encoder, which seems to reduce 2s of the latency. Then speed-preset=ultrafast seems no help on the latency. Anyone can advise how to further reduce the latency on gStreamer pipeline? I am new to gStreamer. Hit lightly if I make any mistakes. :p
>
> Here is my pipeline:
> gst-launch-1.0 ksvideosrc do-timestamp=TRUE device-name="Studio Camera" ! video/x-raw,width=1920,height=1080,framerate=30/1 ! videoconvert ! x264enc bframes=0 key-int-max=45 bitrate=4096 tune=zerolatency speed-preset=ultrafast ! video/x-h264,profile=baseline,stream-format=avc,alignment=au ! kvssink stream-name="ANGELA-Stream" access-key="xxx" secret-key="xxx" aws-region="xxx"
>
> Thank you,
> Angela
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220630/4300b7e6/attachment-0001.htm>