webrtcbin: 4K h264 problems

[James Linder]

> On 11 Oct 2021, at 3:55 pm, Rob Agar via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:
> 
> Hi David
> 
> We're also struggling to get h264 video to work reliably via webrtc, but here on Ubuntu (18.04 & 20.04), even at lower resolutions & bit rate.
> 
> It does sound like a similar situation though - when it occasionally works, the video is stuttery with packet loss reported in chrome://webrtc-internals
> 
> Rob
> 
> On 11/10/2021 06:09, David Ely via gstreamer-devel wrote:
>> We are having issues sending a 4K, H264 stream through WebRTC on Windows (3840x2160, 30hz, 8Mbs). Playback of the video stream stutters or may get stuck. Our application captures the desktop, encodes it to H264 and streams it to Chrome or another of our applications, both of which exhibit the problem. This issue only happens on Windows, CentOS7 and Ubuntu20.04 work (using the same hardware). Lower resolutions (1080P) work across all 3 operations systems. Chrome's webrtc-internals shows the packetsLost or freezeCount increase when the stutter occurs.
>> Points of note:
>> 	• Our GStreamer version is 1.19.1.
>> 	• This was reproduced with two PCs on a local network. The problem won't occur if the sender and receiver are on the same computer.
>> 	• We did some Wireshark captures on the sending PC and it showed some packets aren't making it to the NIC. It appears occasionally a NAL is being truncated.
>> 	• In attempt reduced the scope of the problem, I reproduced the issue in one of the gst-examples (webrtc\sendonly\webrtc-unidirectional-h264.c) by modifying the pipeline to stream from a file (I only tested this on Windows). The file is 30hz, 4K H264 video with a birate of 8 Mbs. The pipeline looks like this
>> 		• filesrc location=C:\tmp\test.mkv ! queue ! matroskademux ! queue ! h264parse ! video/x-h264,alignment=au,stream-format=byte-stream ! rtph264pay config-interval=1 aggregate-mode ! application/x-rtp,media=video,encoding-name=H264,encoding-payload=96 ! webrtcbin
>> 	• Reducing the bitrate alleviates the problem.
>> 	• For reference this is our application's pipeline on Windows
>> 		• dxgiscreencapsrc cursor=true ! video/x-raw,format=BGRA,framerate=30/1 ! videoconvert ! queue ! video/x-raw,format=I420 ! nvh264enc preset=low-latency-hp bitrate=8192 rc-mode=cbr gop-size=90 zerolatency=true ! video/x-h264,framerate=30/1,alignment=au,stream-format=byte-stream ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! webrtcbin
>> As I was able to reproduce the issue in the modified example it points to webrtcbin as the problem.
>> Any help/suggestions to resolve the problem would be greatly appreciated.
>> Thanks
>> David Ely
>> Software Architect

Rob (David) I struggled lots. Finally a bit of clear

1.16 stutters
‘buntu 18.04 .. 21.04 use 1.16 but you can upgrade the gstreamer stuff to 1.18 (which I’ve seen no stutters on except rtsp (all h264)
So your Distro and Version of gstreamer counts lots.

I built gstreamer from git (was really quite painless) using meson as per the wiki. That worked (I got 1.19-2)

James