Problem setting WebRTC connection between GStreamer and FreeSwitch
daniel at poradnik-webmastera.com
daniel at poradnik-webmastera.com
Thu Mar 29 11:20:10 UTC 2018
W dniu 2018-03-29 03:27, Matthew Waters napisał(a):
> On 29/03/18 02:05, daniel at poradnik-webmastera.com wrote:
>> W dniu 2018-03-28 12:14, daniel at poradnik-webmastera.com napisał(a):
>>> W dniu 2018-03-28 10:27, Matthew Waters napisał(a):
>>>> On 28/03/18 00:48, daniel at poradnik-webmastera.com wrote:
>>>>> Hi,
>>>>> I am trying to establish WebRTC connection between GStreamer and
>>>>> FreeSwitch. FreeSwitch itself works - I am able to connect to it
>>>>> using
>>>>> Blink VoIP client, and with WebRTC using Chrome+SIP.js.
>>>>>
>>>>> I started with
>>>>> https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
>>>>>
>>>>> example, and modified it to talk with JS script running on node.js,
>>>>> which uses SIP.js to communicate with FreeSwitch.
>>>>>
>>>>> My pipeline is simplified to use audio only, and no STUN:
>>>>> pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
>>>>> "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay
>>>>> ! "
>>>>> "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
>>>>> &error);
>>>>>
>>>>> This is SDP created by GST, with ICE candidates added:
>>>>>
>>>>> v=0
>>>>> o=- 8118924877098511175 0 IN IP4 0.0.0.0
>>>>> s=-
>>>>> t=0 0
>>>>> a=ice-options:trickle
>>>>> m=audio 9 UDP/TLS/RTP/SAVPF 97
>>>>> c=IN IP4 0.0.0.0
>>>>> a=setup:actpass
>>>>> a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
>>>>> a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
>>>>> a=sendrecv
>>>>> a=rtcp-mux
>>>>> a=rtcp-rsize
>>>>> a=rtpmap:97 OPUS/48000/2
>>>>> a=rtcp-fb:97 nack
>>>>> a=rtcp-fb:97 nack pli
>>>>> a=mid:audio0
>>>>> a=fingerprint:sha-256
>>>>> CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
>>>>>
>>>>> a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
>>>>> a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype
>>>>> active
>>>>> a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host
>>>>> tcptype
>>>>> passive
>>>>> a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
>>>>> a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype
>>>>> active
>>>>> a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host
>>>>> tcptype
>>>>> passive
>>>>> a=end-of-candidates
>>>>>
>>>>> FreeSwitch accepts it, and sends following one in reply:
>>>>>
>>>>> v=0
>>>>> o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
>>>>> s=FreeSWITCH
>>>>> c=IN IP4 192.168.100.20
>>>>> t=0 0
>>>>> a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
>>>>> m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
>>>>> a=rtpmap:97 OPUS/48000/2
>>>>> a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
>>>>> a=rtpmap:101 telephone-event/8000
>>>>> a=ptime:20
>>>>> a=fingerprint:sha-256
>>>>> 95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
>>>>>
>>>>> a=setup:active
>>>>> a=rtcp-mux
>>>>> a=rtcp:19410 IN IP4 192.168.100.20
>>>>
>>>> This a=rtcp line is problematic if FreeSwitch is expecting RTCP
>>>> packets
>>>> to appear on port 19410. RTCP is transmitted through the ICE
>>>> connection
>>>> and thus is not generally signalled through the SDP.
>>>
>>> The same IP and port for RTCP are implied by a=rtcp-mux line above,
>>> so
>>> this line looks redundant for me. Or am I missing something?
>>>
>>>>> a=ice-ufrag:lR5j1yY3DFPhRJNO
>>>>> a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
>>>>> a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
>>>>> generation 0
>>>>> a=end-of-candidates
>>>>> a=ssrc:1792117166 cname:6niZnGNXemS9odOg
>>>>> a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
>>>>> a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
>>>>> a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
>>>>
>>>> Does FreeSwitch require the msid/cname handling because that's
>>>> currently
>>>> not implemented by webrtcbin.
>>>
>>> I do not know yet, I am checking this now.
>>>
>>>>> I send it in set-remote-description as in example code, but GST is
>>>>> not
>>>>> able to establish SRTP session. In Wireshark capture I see that
>>>>> FreeSwitch sent few STUN Binding Requests (GST responded to them)
>>>>> and
>>>>> RTCP packets with Receiver Report and Source description.
>>>>>
>>>>> GST debug logs are very verbose and it was hard for me to find some
>>>>> useful info there. I tried to filter them a bit, and found
>>>>> following
>>>>> entries there:
>>>>>
>>>>> 0:00:01.611202908 542 0x1b5b4a0 WARN structure
>>>>> gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
>>>>> transform to serialize field 'srtp-key' of type 'buffer'
>>>>>
>>>>>
>>>>> 0:00:05.670806044 542 0x1bd1190 DEBUG GST_SCHEDULING
>>>>> gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink>
>>>>> calling
>>>>> chainfunction &gst_proxy_pad_chain_default with buffer buffer:
>>>>> 0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
>>>>> 99:99:99.999999999, size 92, offset none, offset_end none, flags
>>>>> 0x40
>>>>> 0:00:05.670825393 542 0x1bd1190 DEBUG GST_SCHEDULING
>>>>> gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
>>>>> calling chainfunction &sink_chain with buffer buffer:
>>>>> 0x7f6c10005040,
>>>>> pts 0:00:04.151691973, dts 0:00:04.151691973, dur
>>>>> 99:99:99.999999999,
>>>>> size 92, offset none, offset_end none, flags 0x40
>>>>> 0:00:05.670856500 542 0x1bd1190 LOG dtlssrtpdemux
>>>>> gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp
>>>>> packet
>>>>> 0:00:05.671088870 542 0x1bd1190 WARN dtlssrtpdec
>>>>> gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp
>>>>> key
>>>>> available yet
>>>>> 0:00:05.671104066 542 0x1bd1190 WARN srtpdec
>>>>> gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get
>>>>> caps
>>>>> for stream with SSRC 1792117166
>>>>> 0:00:05.671115242 542 0x1bd1190 WARN srtpdec
>>>>> gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer,
>>>>> dropping
>>>>> 0:00:05.671154906 542 0x1bd1190 DEBUG GST_SCHEDULING
>>>>> gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
>>>>> called chainfunction &sink_chain with buffer 0x7f6c10005040,
>>>>> returned ok
>>>>> 0:00:05.671166503 542 0x1bd1190 DEBUG GST_SCHEDULING
>>>>> gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink>
>>>>> called
>>>>> chainfunction &gst_proxy_pad_chain_default with buffer
>>>>> 0x7f6c10005040,
>>>>> returned ok
>>>>
>>>> This usually means that something's not quite right with what
>>>> GStreamer
>>>> expects.
>>>
>>> In the meantime I found
>>> https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
>>>
>>> . It looks that GStreamer for some reason does not perform DTSL
>>> handshake. BTW, it seems that FreeSwitch started its part - in its
>>> log
>>> I found this:
>>>
>>> ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282
>>> [INFO]
>>> switch_rtp.c:3752 Changing audio DTLS state from OFF
>>> to HANDSHAKE
>>>
>>> Here is one of STUN Binding Request packets sent by FreeSwitch. It
>>> was
>>> captured during other test, so its contents does not match SDP pasted
>>> earlier:
>>>
>>> Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568
>>> bits)
>>> Linux cooked capture
>>> Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
>>> User Datagram Protocol, Src Port: 24012, Dst Port: 45557
>>> Session Traversal Utilities for NAT
>>> [Response In: 91]
>>> Message Type: 0x0001 (Binding Request)
>>> Message Length: 132
>>> Message Cookie: 2112a442
>>> Message Transaction ID: 364e4d535879554441354a61
>>> Attributes
>>> USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
>>> Attribute Type: USERNAME (0x0006)
>>> Attribute Length: 49
>>> Username:
>>> cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
>>> Padding: 3
>>> PRIORITY
>>> Attribute Type: PRIORITY (0x0024)
>>> Attribute Length: 4
>>> Priority: 2013266428
>>> SOFTWARE
>>> Attribute Type: SOFTWARE (0x8022)
>>> Attribute Length: 19
>>> Software: FreeSWITCH ( 64bit)
>>> Padding: 1
>>> ICE-CONTROLLED
>>> Attribute Type: ICE-CONTROLLED (0x8029)
>>> Attribute Length: 8
>>> Tie breaker: 3967545348314432
>>> MESSAGE-INTEGRITY
>>> Attribute Type: MESSAGE-INTEGRITY (0x0008)
>>> Attribute Length: 20
>>> HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
>>> FINGERPRINT
>>> Attribute Type: FINGERPRINT (0x8028)
>>> Attribute Length: 4
>>> CRC-32: 0xbd08d97e
>>>
>>> I hope it will help you.
>>
>> I am out of ideas today. It looks that DTLS-SRTP handshake is not
>> started at all. I would expect that GStreamer would initiate it -
>> maybe it also assumes that it has server role and expects that the
>> other side will initiate handshake?
>>
>> I have filtered dtlssrtp-related lines from log and uploaded them
>> here. Please take a look, maybe you will find something there. First
>> entry after sending remove SDP to GST is at 0:00:01.399761438.
>> https://pastebin.com/uMncN6jn
>
> Ah, that's because it errors out :)
>
> 0:00:01.651764557 542 0x1b5b450 ERROR webrtcbin
> gstwebrtcbin.c:2294:_set_description_task:<sendrecv> media 0 is
> missing or contains an empty 'mid' attribute
>
> IIRC, mid is a required field for webrtc.
Thanks for hint! I have manually added "a=mid:audio0" at the end of
FreeStreamer SDP, and things moved forward. However GStreamer started
complaining about another thing:
0:00:04.290276668 9069 0x16ec450 ERROR default
webrtcsdp.c:511:_get_final_direction: Abnormal situation!
I checked source code and it turned out that GStreamer was complaining
about missing direction attribute. I added "a=sendrecv" to FreeStreamer
SDP and audio started going to FS! Hooray!
I checked RFC 4566 and found this note in description of a=sendrecv:
"If none of the attributes "sendonly", "recvonly", "inactive",
and "sendrecv" is present, "sendrecv" SHOULD be assumed as the
default for sessions that are not of the conference type
"broadcast" or "H332" (see below)."
Looks that GStreamer does not follow RFC in this case. I will log a bug
for this.
BTW, what do you think about making GStreamer more robust and making
a=mid optional, especially in case when only one stream (e.g. audio
only) is present?
Thanks for your help!
Regards,
Daniel
>
> Cheers
> -Matt
>
>> Regards,
>> Daniel
>>
>>>
>>> Regards,
>>> Daniel
>>>
>>>>> Any ideas how to fix this?
>>>>
>>>> Depends very much on what FreeSwitch is expecting and producing :).
>>>>
>>>> Cheers
>>>> -Matt
>>>>
>>>>> My FreeSwitch is installed on CentOS from official RPMs, and
>>>>> (mostly)
>>>>> with default config. GStreamer 1.14 is compiled from sources there.
>>>>>
>>>>> Regards,
>>>>> Daniel
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>
>>>>
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list