webrtcbin: 4K h264 problems

[David Ely]

Thanks James and Rob

I thought it was worth mentioning I've also reproduced this issue encoding with VP8.  

Just to reiterate, this works on Centos and Ubuntu (on the same physical pc running Windows).  It’s a Windows only problem. 


Thanks 
David Ely 
Software Architect 

T
 +61 7 3123 7133
E
 david.ely at immersaview.com
W
 www.immersaview.com
A
 13/67 Depot St, Banyo, QLD 4014

This electronic mail transmission is confidential and is intended only for the review of the party to whom it is addressed. 


-----Original Message-----
From: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> On Behalf Of James Linder via gstreamer-devel
Sent: Monday, 11 October 2021 7:35 PM
To: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Cc: James Linder <jam at tigger.ws>
Subject: Re: webrtcbin: 4K h264 problems



> On 11 Oct 2021, at 3:55 pm, Rob Agar via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:
> 
> Hi David
> 
> We're also struggling to get h264 video to work reliably via webrtc, but here on Ubuntu (18.04 & 20.04), even at lower resolutions & bit rate.
> 
> It does sound like a similar situation though - when it occasionally 
> works, the video is stuttery with packet loss reported in 
> chrome://webrtc-internals
> 
> Rob
> 
> On 11/10/2021 06:09, David Ely via gstreamer-devel wrote:
>> We are having issues sending a 4K, H264 stream through WebRTC on Windows (3840x2160, 30hz, 8Mbs). Playback of the video stream stutters or may get stuck. Our application captures the desktop, encodes it to H264 and streams it to Chrome or another of our applications, both of which exhibit the problem. This issue only happens on Windows, CentOS7 and Ubuntu20.04 work (using the same hardware). Lower resolutions (1080P) work across all 3 operations systems. Chrome's webrtc-internals shows the packetsLost or freezeCount increase when the stutter occurs.
>> Points of note:
>> 	• Our GStreamer version is 1.19.1.
>> 	• This was reproduced with two PCs on a local network. The problem won't occur if the sender and receiver are on the same computer.
>> 	• We did some Wireshark captures on the sending PC and it showed some packets aren't making it to the NIC. It appears occasionally a NAL is being truncated.
>> 	• In attempt reduced the scope of the problem, I reproduced the issue in one of the gst-examples (webrtc\sendonly\webrtc-unidirectional-h264.c) by modifying the pipeline to stream from a file (I only tested this on Windows). The file is 30hz, 4K H264 video with a birate of 8 Mbs. The pipeline looks like this
>> 		• filesrc location=C:\tmp\test.mkv ! queue ! matroskademux ! queue ! h264parse ! video/x-h264,alignment=au,stream-format=byte-stream ! rtph264pay config-interval=1 aggregate-mode ! application/x-rtp,media=video,encoding-name=H264,encoding-payload=96 ! webrtcbin
>> 	• Reducing the bitrate alleviates the problem.
>> 	• For reference this is our application's pipeline on Windows
>> 		• dxgiscreencapsrc cursor=true ! 
>> video/x-raw,format=BGRA,framerate=30/1 ! videoconvert ! queue ! video/x-raw,format=I420 ! nvh264enc preset=low-latency-hp bitrate=8192 rc-mode=cbr gop-size=90 zerolatency=true ! video/x-h264,framerate=30/1,alignment=au,stream-format=byte-stream ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! webrtcbin As I was able to reproduce the issue in the modified example it points to webrtcbin as the problem.
>> Any help/suggestions to resolve the problem would be greatly appreciated.
>> Thanks
>> David Ely
>> Software Architect

Rob (David) I struggled lots. Finally a bit of clear

1.16 stutters
‘buntu 18.04 .. 21.04 use 1.16 but you can upgrade the gstreamer stuff to 1.18 (which I’ve seen no stutters on except rtsp (all h264) So your Distro and Version of gstreamer counts lots.

I built gstreamer from git (was really quite painless) using meson as per the wiki. That worked (I got 1.19-2)

James