Can't make webrtcbin connect to H.264 offering clients

Sun Aug 22 12:57:24 UTC 2021

Le dim. 22 août 2021 08 h 48, Neil Young <foreverneilyoung at googlemail.com>
a écrit :

> BTW: I will also be happy to hand out a POC app to everybody willing to
> help. You would only need to have a webrtcbin capable installation of GST
> and Python3. Works out of the box, no strings attached.
>

An answering app, I didn't link it before cause it's demoware. See the
known issue about mdns candidates though, this is a missing feature that
can pause problem.

https://gitlab.collabora.com/showcases/ibc-2019/multistream-server

In general, specially at this time of the year, a lot of folks are on
vacation, leaving other really busy. Expecting someone to look into your
issues for free in such a short timeframe is not realistic.

> Am 22.08.2021 um 14:46 schrieb Neil Young <foreverneilyoung at googlemail.com
> >:
>
> Well, I have published everything I did so far. I confirm, it works if
> webrtc/python is the offerer and it also works as described with an
> incoming offer, if the pipleline is using vp8 instead of H.264.
>
> But it does not no work with H.264. And unless anybody points me to the
> obvious mistake I have made with my handling there is nothing to be taken
> back with my conclusion.
>
> Prove me wrong, if you can. It doesn't help just to say "it works". There
> are very rare sample of answering apps with webrtcbin. And I'm doing
> nothing else then those.
>
>
>
>
> Am 22.08.2021 um 14:41 schrieb Nicolas Dufresne <nicolas at ndufresne.ca>:
>
>
>
> Le dim. 22 août 2021 07 h 45, Neil Young via gstreamer-devel <
> gstreamer-devel at lists.freedesktop.org> a écrit :
>
>> Finally giving up on this. Python, webrtcbin and H.264 offer - this does
>> not work
>>
>
> Hi Neil, I'm sorry this did not work you and no one was available in your
> required delay to freely support you.
>
> This answer is simply to state that this works for others, the webrtcbin
> and rtpbin pipeline are known to be difficult, but not broken (except for
> few missing webrtc features of course).
>
>
>>
>>
>>
>> Am 21.08.2021 um 20:23 schrieb Neil Young <
>> foreverneilyoung at googlemail.com>:
>>
>> I'm hanging on this for a week now, and it seems so easy, but whatever I
>> do, I can't make it work
>>
>> My app is a Python GST app, running on 18.4. It supports two modes:
>> "call" and "answer". In call mode the app immediately issues an H.264 or
>> VP8 OFFER (depending on a setting). This works in 100 % of the cases.
>>
>> What also works is when the app ANSWERs a VP8 offer. Just H.264 OFFERS
>> are not working, even though the SDP answer looks good. I just don't get
>> the video to remote.
>>
>> My app works like so:
>>
>> 1) When the app has a connection to the signaling server it creates a
>> webrtcbin object:
>>
>> def build_webrtcbin(self):
>> self.webrtcbin = Gst.ElementFactory.make("webrtcbin")
>> self.webrtcbin.set_property("bundle-policy", "max-bundle")
>> self.webrtcbin.set_property("stun-server", "
>> stun://stun.l.google.com:19302")
>> self.webrtcbin.connect('on-negotiation-needed', self
>> .on_negotiation_needed)
>> self.webrtcbin.connect('on-ice-candidate', self.on_ice_candidate)
>> self.webrtcbin.connect('pad-added', self.on_pad_added)
>> self.webrtcbin.connect('notify::connection-state', self
>> .on_connection_state)
>> self.webrtcbin.connect('notify::signaling-state', self
>> .on_signaling_state)
>> self.webrtcbin.connect('notify::ice-connection-state', self
>> .on_ice_connection_state)
>> self.webrtcbin.connect('notify::ice-gathering-state', self
>> .on_ice_gathering_state)
>>
>> self.pipeline = Gst.Pipeline.new()
>> self.pipeline.add(self.webrtcbin)
>>
>> Nothing special I guess.
>>
>> 2) When the SDP OFFER comes in, the app does this:
>>
>> 2a) It starts the H.264 pipeline with `start_pipeline`.
>> 2b) It emits the remote description to the local webrtcbin
>>
>> self.webrtcbin.emit('set-remote-description', desc, promise)
>>
>> 2c) It creates an answer calling `self.create_answer()`
>>
>> def create_answer(self):
>> promise = Gst.Promise.new_with_change_func(self.on_answer_created, self.webrtcbin,
>> None)
>> self.webrtcbin.emit('create-answer', None, promise)
>>
>> def on_answer_created(self, promise, _, __):
>> ''' Local WebRTC stack has created an answer '''
>> promise.wait()
>> reply = promise.get_reply()
>> answer = reply.get_value('answer')
>> promise = Gst.Promise.new()
>> self.webrtcbin.emit('set-local-description', answer, promise)
>> promise.interrupt()
>> self.send_sdp_answer(answer) # Forwarding the answer to remote using the
>> signaling server
>>
>> Observations so far:
>>
>> - It doesn't work if I don't set a payload-type which matches a payload
>> type carried by the OFFERER or if I omit the payload-type. With ALL
>> browsers it is sufficient to use the payload-type carried by the first
>> H.264 element in the OFFER
>> - It is not necessary to set the `profile-level-id`
>>
>> The best result I can achieve with this flow is a spinning wheel in the
>> browser. No video.
>>
>> BUT:
>>
>> When I issue a call to `start_pipeline` somewhere shortly after the start
>> (I use the moment, when I have a connection with the signaling server for
>> this) with _exactly_the_same_payload_type_that_will_match_later_, I 100%
>> have a video...
>>
>> This MUST be a side effect, because the log complains about the second
>> `start_pipeline` (warnings, it seems to ignore it).... But it works. Of
>> course this is not a working solution, because
>>
>> a) I would have to know the payload-type beforehand (it doesn't work if I
>> use another payload-type in the first start)
>> b) I would have to parse the incoming SDP for the first H.264 (which is
>> doable)
>>
>> What am I doing wrong?
>>
>>
>>
>>
>> def start_pipeline(self, payload_type):
>> videotestsrc = Gst.ElementFactory.make("videotestsrc", "videotestsrc")
>> videotestsrc.set_property("is-live", True)
>> videotestsrc.set_property("pattern", "smpte")
>>
>> videotestsrc_caps = Gst.caps_from_string("video/x-raw, width=1280,
>> height=720")
>> videotestsrc_caps_filter = Gst.ElementFactory.make("capsfilter")
>> videotestsrc_caps_filter.set_property("caps", videotestsrc_caps)
>>
>> videoconvert = Gst.ElementFactory.make("videoconvert", "videoconvert")
>>
>> x264enc = Gst.ElementFactory.make("x264enc", "x264enc")
>> x264enc.set_property("threads", 4)
>> x264enc_caps = Gst.caps_from_string("video/x-h264,
>> profile=constrained-baseline")
>>
>> x264enc_caps_filter = Gst.ElementFactory.make("capsfilter")
>> x264enc_caps_filter.set_property("caps", x264enc_caps)
>>
>> rtph264pay = Gst.ElementFactory.make("rtph264pay", "rtph264pay")
>> rtph264pay.set_property("config-interval", -1)
>> rtph264pay.set_property("aggregate-mode", "zero-latency")
>>
>> rtph264pay_caps = Gst.caps_from_string("application/x-rtp, media=video,
>> encoding-name=H264")
>> rtph264pay_caps.set_value("payload", payload_type)
>>
>> rtph264pay_caps_filter = Gst.ElementFactory.make("capsfilter")
>> rtph264pay_caps_filter.set_property("caps", rtph264pay_caps)
>>
>> self.pipeline.add(videotestsrc)
>> self.pipeline.add(videotestsrc_caps_filter)
>> self.pipeline.add(videoconvert)
>> self.pipeline.add(x264enc)
>> self.pipeline.add(x264enc_caps_filter)
>> self.pipeline.add(rtph264pay)
>> self.pipeline.add(rtph264pay_caps_filter)
>>
>> Gst.Element.link <http://gst.element.link/>(videotestsrc,
>> videotestsrc_caps_filter)
>> Gst.Element.link <http://gst.element.link/>(videotestsrc_caps_filter,
>> videoconvert)
>> Gst.Element.link <http://gst.element.link/>(videoconvert, x264enc)
>> Gst.Element.link <http://gst.element.link/>(x264enc, x264enc_caps_filter)
>> Gst.Element.link <http://gst.element.link/>(x264enc_caps_filter,
>> rtph264pay)
>> Gst.Element.link <http://gst.element.link/>(rtph264pay,
>> rtph264pay_caps_filter)
>> Gst.Element.link <http://gst.element.link/>(rtph264pay_caps_filter, self
>> .webrtcbin)
>> self.pipeline.set_state(Gst.State.PLAYING)
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210822/aa55560c/attachment-0001.htm>