Problem setting WebRTC connection between GStreamer and FreeSwitch

Matthew Waters ystreet00 at gmail.com
Thu Mar 29 11:47:20 UTC 2018 

On 29/03/18 22:20, daniel at poradnik-webmastera.com wrote:
> W dniu 2018-03-29 03:27, Matthew Waters napisał(a):
>> On 29/03/18 02:05, daniel at poradnik-webmastera.com wrote:
>>> W dniu 2018-03-28 12:14, daniel at poradnik-webmastera.com napisał(a):
>>>> W dniu 2018-03-28 10:27, Matthew Waters napisał(a):
>>>>> On 28/03/18 00:48, daniel at poradnik-webmastera.com wrote:
>>>>>> Hi,
>>>>>> I am trying to establish WebRTC connection between GStreamer and
>>>>>> FreeSwitch. FreeSwitch itself works - I am able to connect to it
>>>>>> using
>>>>>> Blink VoIP client, and with WebRTC using Chrome+SIP.js.
>>>>>>
>>>>>> I started with
>>>>>> https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
>>>>>>
>>>>>>
>>>>>> example, and modified it to talk with JS script running on node.js,
>>>>>> which uses SIP.js to communicate with FreeSwitch.
>>>>>>
>>>>>> My pipeline is simplified to use audio only, and no STUN:
>>>>>>     pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
>>>>>>         "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay
>>>>>> ! "
>>>>>>         "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
>>>>>>         &error);
>>>>>>
>>>>>> This is SDP created by GST, with ICE candidates added:
>>>>>>
>>>>>> v=0
>>>>>> o=- 8118924877098511175 0 IN IP4 0.0.0.0
>>>>>> s=-
>>>>>> t=0 0
>>>>>> a=ice-options:trickle
>>>>>> m=audio 9 UDP/TLS/RTP/SAVPF 97
>>>>>> c=IN IP4 0.0.0.0
>>>>>> a=setup:actpass
>>>>>> a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
>>>>>> a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
>>>>>> a=sendrecv
>>>>>> a=rtcp-mux
>>>>>> a=rtcp-rsize
>>>>>> a=rtpmap:97 OPUS/48000/2
>>>>>> a=rtcp-fb:97 nack
>>>>>> a=rtcp-fb:97 nack pli
>>>>>> a=mid:audio0
>>>>>> a=fingerprint:sha-256
>>>>>> CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
>>>>>>
>>>>>>
>>>>>> a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
>>>>>> a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype
>>>>>> active
>>>>>> a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host
>>>>>> tcptype
>>>>>> passive
>>>>>> a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
>>>>>> a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype
>>>>>> active
>>>>>> a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host
>>>>>> tcptype
>>>>>> passive
>>>>>> a=end-of-candidates
>>>>>>
>>>>>> FreeSwitch accepts it, and sends following one in reply:
>>>>>>
>>>>>> v=0
>>>>>> o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
>>>>>> s=FreeSWITCH
>>>>>> c=IN IP4 192.168.100.20
>>>>>> t=0 0
>>>>>> a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
>>>>>> m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
>>>>>> a=rtpmap:97 OPUS/48000/2
>>>>>> a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
>>>>>> a=rtpmap:101 telephone-event/8000
>>>>>> a=ptime:20
>>>>>> a=fingerprint:sha-256
>>>>>> 95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
>>>>>>
>>>>>>
>>>>>> a=setup:active
>>>>>> a=rtcp-mux
>>>>>> a=rtcp:19410 IN IP4 192.168.100.20
>>>>>
>>>>> This a=rtcp line is problematic if FreeSwitch is expecting RTCP
>>>>> packets
>>>>> to appear on port 19410.  RTCP is transmitted through the ICE
>>>>> connection
>>>>> and thus is not generally signalled through the SDP.
>>>>
>>>> The same IP and port for RTCP are implied by a=rtcp-mux line above, so
>>>> this line looks redundant for me. Or am I missing something?
>>>>
>>>>>> a=ice-ufrag:lR5j1yY3DFPhRJNO
>>>>>> a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
>>>>>> a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
>>>>>> generation 0
>>>>>> a=end-of-candidates
>>>>>> a=ssrc:1792117166 cname:6niZnGNXemS9odOg
>>>>>> a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
>>>>>> a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
>>>>>> a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
>>>>>
>>>>> Does FreeSwitch require the msid/cname handling because that's
>>>>> currently
>>>>> not implemented by webrtcbin.
>>>>
>>>> I do not know yet, I am checking this now.
>>>>
>>>>>> I send it in set-remote-description as in example code, but GST
>>>>>> is not
>>>>>> able to establish SRTP session. In Wireshark capture I see that
>>>>>> FreeSwitch sent few STUN Binding Requests (GST responded to them)
>>>>>> and
>>>>>> RTCP packets with Receiver Report and Source description.
>>>>>>
>>>>>> GST debug logs are very verbose and it was hard for me to find some
>>>>>> useful info there. I tried to filter them a bit, and found following
>>>>>> entries there:
>>>>>>
>>>>>> 0:00:01.611202908   542      0x1b5b4a0 WARN               structure
>>>>>> gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
>>>>>> transform to serialize field 'srtp-key' of type 'buffer'
>>>>>>
>>>>>>
>>>>>> 0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
>>>>>> gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink>
>>>>>> calling
>>>>>> chainfunction &gst_proxy_pad_chain_default with buffer buffer:
>>>>>> 0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
>>>>>> 99:99:99.999999999, size 92, offset none, offset_end none, flags
>>>>>> 0x40
>>>>>> 0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
>>>>>> gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
>>>>>> calling chainfunction &sink_chain with buffer buffer:
>>>>>> 0x7f6c10005040,
>>>>>> pts 0:00:04.151691973, dts 0:00:04.151691973, dur
>>>>>> 99:99:99.999999999,
>>>>>> size 92, offset none, offset_end none, flags 0x40
>>>>>> 0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
>>>>>> gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp
>>>>>> packet
>>>>>> 0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
>>>>>> gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no
>>>>>> srtp key
>>>>>> available yet
>>>>>> 0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
>>>>>> gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get
>>>>>> caps
>>>>>> for stream with SSRC 1792117166
>>>>>> 0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
>>>>>> gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer,
>>>>>> dropping
>>>>>> 0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
>>>>>> gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
>>>>>> called chainfunction &sink_chain with buffer 0x7f6c10005040,
>>>>>> returned ok
>>>>>> 0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
>>>>>> gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink>
>>>>>> called
>>>>>> chainfunction &gst_proxy_pad_chain_default with buffer
>>>>>> 0x7f6c10005040,
>>>>>> returned ok
>>>>>
>>>>> This usually means that something's not quite right with what
>>>>> GStreamer
>>>>> expects.
>>>>
>>>> In the meantime I found
>>>> https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
>>>>
>>>>
>>>> . It looks that GStreamer for some reason does not perform DTSL
>>>> handshake. BTW, it seems that FreeSwitch started its part - in its log
>>>> I found this:
>>>>
>>>> ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
>>>> switch_rtp.c:3752 Changing audio DTLS state from OFF
>>>>  to HANDSHAKE
>>>>
>>>> Here is one of STUN Binding Request packets sent by FreeSwitch. It was
>>>> captured during other test, so its contents does not match SDP pasted
>>>> earlier:
>>>>
>>>> Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568
>>>> bits)
>>>> Linux cooked capture
>>>> Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
>>>> User Datagram Protocol, Src Port: 24012, Dst Port: 45557
>>>> Session Traversal Utilities for NAT
>>>>     [Response In: 91]
>>>>     Message Type: 0x0001 (Binding Request)
>>>>     Message Length: 132
>>>>     Message Cookie: 2112a442
>>>>     Message Transaction ID: 364e4d535879554441354a61
>>>>     Attributes
>>>>         USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
>>>>             Attribute Type: USERNAME (0x0006)
>>>>             Attribute Length: 49
>>>>             Username:
>>>> cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
>>>>             Padding: 3
>>>>         PRIORITY
>>>>             Attribute Type: PRIORITY (0x0024)
>>>>             Attribute Length: 4
>>>>             Priority: 2013266428
>>>>         SOFTWARE
>>>>             Attribute Type: SOFTWARE (0x8022)
>>>>             Attribute Length: 19
>>>>             Software: FreeSWITCH ( 64bit)
>>>>             Padding: 1
>>>>         ICE-CONTROLLED
>>>>             Attribute Type: ICE-CONTROLLED (0x8029)
>>>>             Attribute Length: 8
>>>>             Tie breaker: 3967545348314432
>>>>         MESSAGE-INTEGRITY
>>>>             Attribute Type: MESSAGE-INTEGRITY (0x0008)
>>>>             Attribute Length: 20
>>>>             HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
>>>>         FINGERPRINT
>>>>             Attribute Type: FINGERPRINT (0x8028)
>>>>             Attribute Length: 4
>>>>             CRC-32: 0xbd08d97e
>>>>
>>>> I hope it will help you.
>>>
>>> I am out of ideas today. It looks that DTLS-SRTP handshake is not
>>> started at all. I would expect that GStreamer would initiate it -
>>> maybe it also assumes that it has server role and expects that the
>>> other side will initiate handshake?
>>>
>>> I have filtered dtlssrtp-related lines from log and uploaded them
>>> here. Please take a look, maybe you will find something there. First
>>> entry after sending remove SDP to GST is at 0:00:01.399761438.
>>> https://pastebin.com/uMncN6jn
>>
>> Ah, that's because it errors out :)
>>
>> 0:00:01.651764557   542      0x1b5b450 ERROR              webrtcbin
>> gstwebrtcbin.c:2294:_set_description_task:<sendrecv> media 0 is
>> missing or contains an empty 'mid' attribute
>>
>> IIRC, mid is a required field for webrtc.
>
> Thanks for hint! I have manually added "a=mid:audio0" at the end of
> FreeStreamer SDP, and things moved forward. However GStreamer started
> complaining about another thing:
>
> 0:00:04.290276668  9069      0x16ec450 ERROR                default
> webrtcsdp.c:511:_get_final_direction: Abnormal situation!
>
> I checked source code and it turned out that GStreamer was complaining
> about missing direction attribute. I added "a=sendrecv" to
> FreeStreamer SDP and audio started going to FS! Hooray!

Awesome!

> I checked RFC 4566 and found this note in description of a=sendrecv:
> "If none of the attributes "sendonly", "recvonly", "inactive",
> and "sendrecv" is present, "sendrecv" SHOULD be assumed as the
> default for sessions that are not of the conference type
> "broadcast" or "H332" (see below)."
>
> Looks that GStreamer does not follow RFC in this case. I will log a
> bug for this.

The RFC we're following is JSEP
(https://tools.ietf.org/html/draft-ietf-rtcweb-jsep-24) which says that
for WebRTC the offerer must add mid and direction sdp attributes.

That being said, I don't think that taking a missing direction attribute
to mean sendrecv is going to cause obvious problems.

> BTW, what do you think about making GStreamer more robust and making
> a=mid optional, especially in case when only one stream (e.g. audio
> only) is present?

Could probably be done.  I don't think mid is required for anything
major until you start RTP bundling (which we don't support yet).

Cheers
-Matt

> Thanks for your help!
>
> Regards,
> Daniel
>
>>
>> Cheers
>> -Matt
>>
>>> Regards,
>>> Daniel
>>>
>>>>
>>>> Regards,
>>>> Daniel
>>>>
>>>>>> Any ideas how to fix this?
>>>>>
>>>>> Depends very much on what FreeSwitch is expecting and producing :).
>>>>>
>>>>> Cheers
>>>>> -Matt
>>>>>
>>>>>> My FreeSwitch is installed on CentOS from official RPMs, and
>>>>>> (mostly)
>>>>>> with default config. GStreamer 1.14 is compiled from sources there.
>>>>>>
>>>>>> Regards,
>>>>>> Daniel
>>>>>> _______________________________________________
>>>>>> gstreamer-devel mailing list
>>>>>> gstreamer-devel at lists.freedesktop.org
>>>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180329/e9dc8a2f/attachment-0001.sig>

More information about the gstreamer-devel mailing list