Can't make webrtcbin connect to H.264 offering clients

[Neil Young]

Finally giving up on this. Python, webrtcbin and H.264 offer - this does not work



> Am 21.08.2021 um 20:23 schrieb Neil Young <foreverneilyoung at googlemail.com>:
> 
> I'm hanging on this for a week now, and it seems so easy, but whatever I do, I can't make it work
> 
> My app is a Python GST app, running on 18.4. It supports two modes: "call" and "answer". In call mode the app immediately issues an H.264 or VP8 OFFER (depending on a setting). This works in 100 % of the cases. 
> 
> What also works is when the app ANSWERs a VP8 offer. Just H.264 OFFERS are not working, even though the SDP answer looks good. I just don't get the video to remote.
> 
> My app works like so:
> 
> 1) When the app has a connection to the signaling server it creates a webrtcbin object:
> 
>     def build_webrtcbin(self):
>         self.webrtcbin = Gst.ElementFactory.make("webrtcbin")
>         self.webrtcbin.set_property("bundle-policy", "max-bundle")
>         self.webrtcbin.set_property("stun-server", "stun://stun.l.google.com:19302 <stun://stun.l.google.com:19302>")
>         self.webrtcbin.connect('on-negotiation-needed', self.on_negotiation_needed)
>         self.webrtcbin.connect('on-ice-candidate', self.on_ice_candidate)
>         self.webrtcbin.connect('pad-added', self.on_pad_added)
>         self.webrtcbin.connect('notify::connection-state', self.on_connection_state)
>         self.webrtcbin.connect('notify::signaling-state', self.on_signaling_state)
>         self.webrtcbin.connect('notify::ice-connection-state', self.on_ice_connection_state)
>         self.webrtcbin.connect('notify::ice-gathering-state', self.on_ice_gathering_state)
> 
>         self.pipeline = Gst.Pipeline.new()
>         self.pipeline.add(self.webrtcbin)
> 
> Nothing special I guess.
> 
> 2) When the SDP OFFER comes in, the app does this:
> 
> 2a) It starts the H.264 pipeline with `start_pipeline`. 
> 2b) It emits the remote description to the local webrtcbin
> 
> self.webrtcbin.emit('set-remote-description', desc, promise)
> 
> 2c) It creates an answer calling `self.create_answer()`
> 
>     def create_answer(self):
>         promise = Gst.Promise.new_with_change_func(self.on_answer_created, self.webrtcbin, None)
>         self.webrtcbin.emit('create-answer', None, promise)
>     
>     def on_answer_created(self, promise, _, __):
>         ''' Local WebRTC stack has created an answer '''
>         promise.wait()
>         reply = promise.get_reply()
>         answer = reply.get_value('answer')
>         promise = Gst.Promise.new()
>         self.webrtcbin.emit('set-local-description', answer, promise)
>         promise.interrupt()
>         self.send_sdp_answer(answer) # Forwarding the answer to remote using the signaling server
> 
> Observations so far:
> 
> - It doesn't work if I don't set a payload-type which matches a payload type carried by the OFFERER or if I omit the payload-type. With ALL browsers it is sufficient to use the payload-type carried by the first H.264 element in the OFFER
> - It is not necessary to set the `profile-level-id`
> 
> The best result I can achieve with this flow is a spinning wheel in the browser. No video.
> 
> BUT: 
> 
> When I issue a call to `start_pipeline` somewhere shortly after the start (I use the moment, when I have a connection with the signaling server for this) with _exactly_the_same_payload_type_that_will_match_later_, I 100% have a video...
> 
> This MUST be a side effect, because the log complains about the second `start_pipeline` (warnings, it seems to ignore it).... But it works. Of course this is not a working solution, because
> 
> a) I would have to know the payload-type beforehand (it doesn't work if I use another payload-type in the first start)
> b) I would have to parse the incoming SDP for the first H.264 (which is doable)
> 
> What am I doing wrong?
> 
> 
> 
> 
>     def start_pipeline(self, payload_type):
>         videotestsrc = Gst.ElementFactory.make("videotestsrc", "videotestsrc")
>         videotestsrc.set_property("is-live", True)
>         videotestsrc.set_property("pattern", "smpte")
> 
>         videotestsrc_caps = Gst.caps_from_string("video/x-raw, width=1280, height=720")
>       
>         videotestsrc_caps_filter = Gst.ElementFactory.make("capsfilter")
>         videotestsrc_caps_filter.set_property("caps", videotestsrc_caps)
> 
>         videoconvert = Gst.ElementFactory.make("videoconvert", "videoconvert")
> 
>         x264enc = Gst.ElementFactory.make("x264enc", "x264enc")
>         x264enc.set_property("threads", 4)
>     
>         x264enc_caps = Gst.caps_from_string("video/x-h264, profile=constrained-baseline")
> 
>         x264enc_caps_filter = Gst.ElementFactory.make("capsfilter")
>         x264enc_caps_filter.set_property("caps", x264enc_caps)
> 
>         rtph264pay = Gst.ElementFactory.make("rtph264pay", "rtph264pay")
>         rtph264pay.set_property("config-interval", -1)
>         rtph264pay.set_property("aggregate-mode", "zero-latency")
> 
>         rtph264pay_caps = Gst.caps_from_string("application/x-rtp, media=video, encoding-name=H264")
>         rtph264pay_caps.set_value("payload", payload_type)
> 
>         rtph264pay_caps_filter = Gst.ElementFactory.make("capsfilter")
>         rtph264pay_caps_filter.set_property("caps", rtph264pay_caps)
> 
>         self.pipeline.add(videotestsrc)
>         self.pipeline.add(videotestsrc_caps_filter)
>         self.pipeline.add(videoconvert)
>         self.pipeline.add(x264enc)
>         self.pipeline.add(x264enc_caps_filter)
>         self.pipeline.add(rtph264pay)
>         self.pipeline.add(rtph264pay_caps_filter)
> 
>         Gst.Element.link <http://gst.element.link/>(videotestsrc, videotestsrc_caps_filter)
>         Gst.Element.link <http://gst.element.link/>(videotestsrc_caps_filter, videoconvert)
>         Gst.Element.link <http://gst.element.link/>(videoconvert, x264enc)
>         Gst.Element.link <http://gst.element.link/>(x264enc, x264enc_caps_filter)
>         Gst.Element.link <http://gst.element.link/>(x264enc_caps_filter, rtph264pay)
>         Gst.Element.link <http://gst.element.link/>(rtph264pay, rtph264pay_caps_filter)
>         Gst.Element.link <http://gst.element.link/>(rtph264pay_caps_filter, self.webrtcbin)
>         self.pipeline.set_state(Gst.State.PLAYING)
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210822/20bd51bd/attachment-0001.htm>