WebRTC generate invalid rtpmap for H264
Peter Biro
pettair at gmail.com
Sat Apr 10 09:49:36 UTC 2021
ohh gosh my bad, user error.
So originally I had a 'producer' pipeline:
nvarguscamerasrc sensor-id=<DCL_SENSOR_ID> sensor-mode=0 gainrange="1 16" ispdigitalgainrange="1 1"
! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)20/1
! nvvidconv ! textoverlay name=text_overlay ! video/x-raw,format=I420
! nvvidconv ! nvv4l2h264enc bitrate=8000000 control-rate=1 insert-sps-pps=true ! rtph264pay mtu=1400 ! udpsink auto-multicast=true clients=<clinets>
Which multicasted the RTP stream to clients and on the client side the WebRTC pipeline looked like this:
udpsrc multicast-group=<source> auto-multicast=true port=<port>
! application/x-rtp,media=video,encoding-name=H264,payload=96
! webrtcbin name=webrtc_sink stun-server='stun://10.51.255.237:3478' latency=0
But for testing purposes I tried to simplify the testcase and move the two parts into one pipeline, but somehow I did not applied the changes so the client still got the stream from the network.. My bad. Sorry for the confusion!
After adding the clock-rate=90000 property on the client side:
udpsrc multicast-group=<source> auto-multicast=true port=<port>
! application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000
! webrtcbin name=webrtc_sink stun-server='stun://10.51.255.237:3478' latency=0
The SDP looks much better:
v=0
o=- 8174947542390535236 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS webrtc_sink
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:9I6XsU/+eajaGQOZBYqjgcJp5/PiMRc9
a=ice-pwd:CJJO5TCzJSivnaaLU2b/ohif4OXj+fBj
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=mid:video0
a=fingerprint:sha-256 F9:13:FF:68:E4:83:59:A0:49:A3:5D:E1:BA:CD:15:0E:DE:95:14:0D:F5:F0:88:7A:63:28:D7:CD:0A:7E:C4:CD
But chrome still cannot play WebRTC video on the frontend side, when I compare the two SDPs (the one from the merged into one pipeline and the one when the RTP stream is coming from the UDP sink)
a=framerate:20
a=fmtp:96 packetization-mode=1;profile-level-id=424015;sprop-parameter-sets=Z0JAKJWgHgCJ+VA=,aM48gA==
a=ssrc:1897942919 msid:user2169902196 at host-6f4d07ad webrtctransceiver0
a=ssrc:1897942919 cname:user2169902196 at host-6f4d07ad
So tried to add them to the properties:
udpsrc multicast-group=< source > auto-multicast=true port=<port>
! application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000,framerate=20,packetization-mode=1,profile-level-id=424015,sprop-parameter-sets="Z0JAKJWgHgCJ+VA\=\,aM48gA\=\="
! webrtcbin name=webrtc_sink stun-server='stun://10.51.255.237:3478' latency=0
But only the 'sprop-parameter-sets' property finds its way to the SDP, so the final (still not working) SDP:
v=0
o=- 8318672537835092582 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS webrtc_sink
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:wb9zs9ZzaiIUHHDrZSoJk7zqM7IEKUpD
a=ice-pwd:6OaD73grf0TKP5l05/UZ2BUbNx33+SDU
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=fmtp:96 sprop-parameter-sets=Z0JAKJWgHgCJ+VA=,aM48gA==
a=mid:video0
a=fingerprint:sha-256 71:00:FF:12:E9:E8:F0:9F:54:0B:5C:B7:52:9E:41:4C:79:A5:8A:A9:07:49:C8:BF:CC:4C:4D:5A:86:4A:06:FE
How can I add the missing parameters to the SDP? Also Im not sure if I do any good by just inserting them (for me sprop-parameter-sets looks like a random thing) so Im not sure it profile-level-id and sprop-parameter-sets is a static thing or should be a variable.
Thank you for the help!
Bests,
Peter
Just for the reference the working SDP:
v=0
o=- 7463942138372302405 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS webrtc_sink
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:9oG0AHRZpwiykWEsKz1YRInr6Vbu+SYJ
a=ice-pwd:QQYgqeYA2ckB2jCcCXKdmrcSS7Sb4z+O
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=framerate:20
a=fmtp:96 packetization-mode=1;profile-level-id=424015;sprop-parameter-sets=Z0JAKJWgHgCJ+VA=,aM48gA==
a=ssrc:1897942919 msid:user2169902196 at host-6f4d07ad webrtctransceiver0
a=ssrc:1897942919 cname:user2169902196 at host-6f4d07ad
a=mid:video0
a=fingerprint:sha-256 72:5D:78:02:41:8B:FA:07:77:DB:EB:23:05:E9:3C:BF:D3:59:0C:01:75:24:9C:51:AE:33:9E:27:A3:98:B5:19
> On 2021. Apr 9., at 20:13, Olivier Crête <olivier.crete at collabora.com> wrote:
>
> Hi,
>
> Oh right, you're using webrtcbin.. Erm, you can enable the webrtcbin debug category and it should print the caps that it gets.
>
> Olivier
>
> On Fri, 2021-04-09 at 20:02 +0200, Peter Biro wrote:
>> Hi,
>>
>> Im not sure if if can run it from terminal (I dont know how to do the signalling that way) but I can setup debug logging if it that ok?
>>
>> I dont set any codec preferences on the webrtc.
>>
>> Strange thing this is only appears if I use H264, I just tried to run the very same pipeline with VP8 codec the SDP looks alright (but unfortunately I have to use H264 so this wont be a workaround for my use case):
>>
>> v=0
>> o=- 711117265011778833 0 IN IP4 0.0.0.0
>> s=-
>> t=0 0
>> a=ice-options:trickle
>> a=msid-semantic:WMS webrtc_sink
>> m=video 9 UDP/TLS/RTP/SAVPF 96
>> c=IN IP4 0.0.0.0
>> a=setup:actpass
>> a=ice-ufrag:Qfsq+80LvU2YdnPZzVjuA4/UrODXoKrN
>> a=ice-pwd:HWOOU8kI1droNmky2tjnGekC1NquWNmH
>> a=sendrecv
>> a=rtcp-mux
>> a=rtcp-rsize
>> a=rtpmap:96 VP8/90000
>> a=rtcp-fb:96 nack pli
>> a=ssrc:2035319666 msid:user1666440897 at host-b40b4b48 webrtctransceiver0
>> a=ssrc:2035319666 cname:user1666440897 at host-b40b4b48
>> a=mid:video0
>> a=fingerprint:sha-256 C2:C3:5D:14:8F:06:44:8A:0E:A8:76:4B:D8:46:5D:BF:31:73:F1:CB:32:F7:8B:F8:F5:D1:D9:58:1D:3C:5D:05
>>
>> Thanks,
>> Peter
>>
>>> On 2021. Apr 9., at 19:52, Olivier Crête <olivier.crete at collabora.com <mailto:olivier.crete at collabora.com>> wrote:
>>>
>>> Hi,
>>>
>>> On Fri, 2021-04-09 at 19:22 +0200, Peter Biro wrote:
>>>> Hi all,
>>>>
>>>> I found a pretty strange issue webrtcbin generates the following initial SDP offer with a negative clock rate:
>>>>
>>>> v=0
>>>> o=- 5716970113760455593 0 IN IP4 0.0.0.0
>>>> s=-
>>>> t=0 0
>>>> a=ice-options:trickle
>>>> a=msid-semantic:WMS webrtc_sink
>>>> m=video 9 UDP/TLS/RTP/SAVPF 96
>>>> c=IN IP4 0.0.0.0
>>>> a=setup:actpass
>>>> a=ice-ufrag:0ybqQZNeS9CznnGxJxW/BpKyapc+CXsL
>>>> a=ice-pwd:ffvVDjwSJ7/+AAdh3MjUMMhz91xwZZGI
>>>> a=sendrecv
>>>> a=rtcp-mux
>>>> a=rtcp-rsize
>>>> a=rtpmap:96 H264/-1784864736
>>>> a=rtcp-fb:96 nack pli
>>>> a=mid:video0
>>>> a=fingerprint:sha-256 9B:11:48:BE:41:FF:CA:FE:62:B1:98:07:7B:E6:80:2A:56:9A:56:DC:A2:60:81:27:DE:EC:F5:76:C1:C4:39:51
>>>>
>>>> Actually it is more like a random number it changes in every session setup. However as I see it should be set to 9000.
>>>>
>>>> My pipleine is pretty simple:
>>>>
>>>> nvarguscamerasrc sensor-id=0 sensor-mode=0 gainrange="1 16" ispdigitalgainrange="1 1"
>>>> ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
>>>> ! nvvidconv ! nvv4l2h264enc bitrate=8000000 control-rate=1 insert-sps-pps=true ! rtph264pay mtu=1400
>>>> ! webrtcbin name=webrtc_sink latency=0
>>>>
>>>> And on the application side Im handling the singling.
>>>>
>>>> Do you have any idea how to fix this? What I am missing here?
>>>
>>> This is definitely a bug in webrtcbin, it shouldn't be generating an SDP with an invalid clock rate.
>>>
>>> Can you run it with "gst-launch-1.0 -v" to see wahtr caps the payloaders set on webrtcbin ?
>>>
>>> Do you set any codec preferences on webrtcbin ?
>>>
>>> Olivier
>>>
>>>>
>>>> Thanks,
>>>> Peter
>>>>
>>>>
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>>>
>>> --
>>> Olivier Crête
>>> olivier.crete at collabora.com <mailto:olivier.crete at collabora.com>
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>
> --
> Olivier Crête
> olivier.crete at collabora.com <mailto:olivier.crete at collabora.com>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210410/cf684e3b/attachment-0001.htm>
More information about the gstreamer-devel
mailing list