<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Finally giving up on this. Python, webrtcbin and H.264 offer - this does not work<div class=""><br class=""></div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">Am 21.08.2021 um 20:23 schrieb Neil Young <<a href="mailto:foreverneilyoung@googlemail.com" class="">foreverneilyoung@googlemail.com</a>>:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I'm hanging on this for a week now, and it seems so easy, but whatever I do, I can't make it work<div class=""><br class=""></div><div class="">My app is a Python GST app, running on 18.4. It supports two modes: "call" and "answer". In call mode the app immediately issues an H.264 or VP8 OFFER (depending on a setting). This works in 100 % of the cases. </div><div class=""><br class=""></div><div class="">What also works is when the app ANSWERs a VP8 offer. Just H.264 OFFERS are not working, even though the SDP answer looks good. I just don't get the video to remote.</div><div class=""><br class=""></div><div class="">My app works like so:</div><div class=""><br class=""></div><div class="">1) When the app has a connection to the signaling server it creates a webrtcbin object:</div><div class=""><br class=""></div><div class=""><div style="background-color: rgb(255, 255, 255); font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; line-height: 21px; white-space: pre;" class=""><div class=""> <span style="color: #210de5;font-weight: bold;" class="">def</span> <span style="color: #1200bb;" class="">build_webrtcbin</span>(<span style="color: #ab3c11;font-style: italic;" class="">self</span>):</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"webrtcbin"</span>)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.set_property(<span style="color: #ba2500;" class="">"bundle-policy"</span>, <span style="color: #ba2500;" class="">"max-bundle"</span>)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.set_property(<span style="color: #ba2500;" class="">"stun-server"</span>, <span style="color: #ba2500;" class="">"<a href="stun://stun.l.google.com:19302" class="">stun://stun.l.google.com:19302</a>"</span>)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'on-negotiation-needed'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_negotiation_needed)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'on-ice-candidate'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_ice_candidate)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'pad-added'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_pad_added)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'notify::connection-state'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_connection_state)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'notify::signaling-state'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_signaling_state)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'notify::ice-connection-state'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_ice_connection_state)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.connect(<span style="color: #d126d1;" class="">'notify::ice-gathering-state'</span>, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_ice_gathering_state)</div><br class=""><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline <span style="color: #7f0055;" class="">=</span> Gst.Pipeline.new()</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(<span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin)</div><div class=""><br class=""></div><div class="">Nothing special I guess.</div><br class=""></div></div><div style="background-color: rgb(255, 255, 255); line-height: 21px;" class=""><div style="font-family: Verdana; white-space: normal;" class="">2) When the SDP OFFER comes in, the app does this:</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">2a) It starts the H.264 pipeline with `start_pipeline`. </div><div style="font-family: Verdana; white-space: normal;" class="">2b) It emits the remote description to the local webrtcbin</div><div class=""><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; white-space: pre; line-height: 21px;" class=""><span style="color: #7f0055;font-weight: bold;" class=""><br class=""></span></div><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; white-space: pre; line-height: 21px;" class=""><span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.emit(<span style="color: #d126d1;" class="">'set-remote-description'</span>, desc, promise)</div><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; white-space: pre; line-height: 21px;" class=""><br class=""></div><div style="line-height: 21px;" class="">2c) It creates an answer calling `self.create_answer()`</div><div style="line-height: 21px;" class=""><br class=""></div><div style="line-height: 21px;" class=""><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; line-height: 21px; white-space: pre;" class=""><div class=""> <span style="color: #210de5;font-weight: bold;" class="">def</span> <span style="color: #1200bb;" class="">create_answer</span>(<span style="color: #ab3c11;font-style: italic;" class="">self</span>):</div><div class=""> promise <span style="color: #7f0055;" class="">=</span> Gst.Promise.new_with_change_func(<span style="color: #7f0055;font-weight: bold;" class="">self</span>.on_answer_created, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin, <span style="color: #0a9dff;" class="">None</span>)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.emit(<span style="color: #d126d1;" class="">'create-answer'</span>, <span style="color: #0a9dff;" class="">None</span>, promise)</div><div style="line-height: 21px;" class=""><div class=""> </div><div class=""><span style="color: #210de5;font-weight: bold;" class=""> def</span> <span style="color: #1200bb;" class="">on_answer_created</span>(<span style="color: #ab3c11;font-style: italic;" class="">self</span>, <span style="color: #ab3c11;font-style: italic;" class="">promise</span>, <span style="color: #ab3c11;font-style: italic;" class="">_</span>, <span style="color: #ab3c11;font-style: italic;" class="">__</span>):</div><div class=""> <span style="color: #d126d1;" class="">''' Local WebRTC stack has created an answer '''</span></div><div class=""> promise.wait()</div><div class=""> reply <span style="color: #7f0055;" class="">=</span> promise.get_reply()</div><div class=""> answer <span style="color: #7f0055;" class="">=</span> reply.get_value(<span style="color: #d126d1;" class="">'answer'</span>)</div><div class=""> promise <span style="color: rgb(127, 0, 85);" class="">=</span> Gst.Promise.new()</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin.emit(<span style="color: #d126d1;" class="">'set-local-description'</span>, answer, promise)</div><div class=""> promise.interrupt()</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.send_sdp_answer(answer) # Forwarding the answer to remote using the signaling server</div></div></div></div><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; white-space: pre; line-height: 21px;" class=""><br class=""></div></div><div style="font-family: Verdana; white-space: normal;" class="">Observations so far:</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">- It doesn't work if I don't set a payload-type which matches a payload type carried by the OFFERER or if I omit the payload-type. With ALL browsers it is sufficient to use the payload-type carried by the first H.264 element in the OFFER</div><div style="font-family: Verdana; white-space: normal;" class="">- It is not necessary to set the `profile-level-id`</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">The best result I can achieve with this flow is a spinning wheel in the browser. No video.</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">BUT: </div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">When I issue a call to `start_pipeline` somewhere shortly after the start (I use the moment, when I have a connection with the signaling server for this) with _exactly_the_same_payload_type_that_will_match_later_, I 100% have a video...</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">This MUST be a side effect, because the log complains about the second `start_pipeline` (warnings, it seems to ignore it).... But it works. Of course this is not a working solution, because</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">a) I would have to know the payload-type beforehand (it doesn't work if I use another payload-type in the first start)</div><div style="font-family: Verdana; white-space: normal;" class="">b) I would have to parse the incoming SDP for the first H.264 (which is doable)</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class="">What am I doing wrong?</div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class=""><br class=""></div><div style="font-family: Verdana; white-space: normal;" class=""><div style="font-family: "Fira Code", Menlo, Monaco, "Courier New", monospace, Menlo, Monaco, "Courier New", monospace; line-height: 21px; white-space: pre;" class=""><div class=""> <span style="color: #210de5;font-weight: bold;" class="">def</span> <span style="color: #1200bb;" class="">start_pipeline</span>(<span style="color: #ab3c11;font-style: italic;" class="">self</span>, <span style="color: #ab3c11;font-style: italic;" class="">payload_type</span>):</div><div class=""> videotestsrc <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"videotestsrc"</span>, <span style="color: #ba2500;" class="">"videotestsrc"</span>)</div><div class=""> videotestsrc.set_property(<span style="color: #ba2500;" class="">"is-live"</span>, <span style="color: #0a9dff;" class="">True</span>)</div><div class=""> videotestsrc.set_property(<span style="color: #ba2500;" class="">"pattern"</span>, <span style="color: #ba2500;" class="">"smpte"</span>)</div><br class=""><div class=""> videotestsrc_caps <span style="color: #7f0055;" class="">=</span> Gst.caps_from_string(<span style="color: #ba2500;" class="">"video/x-raw, width=1280, height=720"</span>)</div><div class=""> </div><div class=""> videotestsrc_caps_filter <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"capsfilter"</span>)</div><div class=""> videotestsrc_caps_filter.set_property(<span style="color: #ba2500;" class="">"caps"</span>, videotestsrc_caps)</div><br class=""><div class=""> videoconvert <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"videoconvert"</span>, <span style="color: #ba2500;" class="">"videoconvert"</span>)</div><br class=""><div class=""> x264enc <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"x264enc"</span>, <span style="color: #ba2500;" class="">"x264enc"</span>)</div><div class=""> x264enc.set_property(<span style="color: #ba2500;" class="">"threads"</span>, <span style="color: #ff8000;" class="">4</span>)</div><div class=""> </div><div class=""> x264enc_caps <span style="color: #7f0055;" class="">=</span> Gst.caps_from_string(<span style="color: #ba2500;" class="">"video/x-h264, profile=constrained-baseline"</span>)</div><br class=""><div class=""> x264enc_caps_filter <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"capsfilter"</span>)</div><div class=""> x264enc_caps_filter.set_property(<span style="color: #ba2500;" class="">"caps"</span>, x264enc_caps)</div><br class=""><div class=""> rtph264pay <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"rtph264pay"</span>, <span style="color: #ba2500;" class="">"rtph264pay"</span>)</div><div class=""> rtph264pay.set_property(<span style="color: #ba2500;" class="">"config-interval"</span>, <span style="color: #098eff;font-weight: bold;" class="">-</span><span style="color: #ff8000;" class="">1</span>)</div><div class=""> rtph264pay.set_property(<span style="color: #ba2500;" class="">"aggregate-mode"</span>, <span style="color: #ba2500;" class="">"zero-latency"</span>)</div><br class=""><div class=""> rtph264pay_caps <span style="color: #7f0055;" class="">=</span> Gst.caps_from_string(<span style="color: #ba2500;" class="">"application/x-rtp, media=video, encoding-name=H264"</span>)</div><div class=""> rtph264pay_caps.set_value(<span style="color: #ba2500;" class="">"payload"</span>, payload_type)</div><br class=""><div class=""> rtph264pay_caps_filter <span style="color: #7f0055;" class="">=</span> Gst.ElementFactory.make(<span style="color: #ba2500;" class="">"capsfilter"</span>)</div><div class=""> rtph264pay_caps_filter.set_property(<span style="color: #ba2500;" class="">"caps"</span>, rtph264pay_caps)</div><br class=""><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(videotestsrc)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(videotestsrc_caps_filter)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(videoconvert)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(x264enc)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(x264enc_caps_filter)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(rtph264pay)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.add(rtph264pay_caps_filter)</div><br class=""><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(videotestsrc, videotestsrc_caps_filter)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(videotestsrc_caps_filter, videoconvert)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(videoconvert, x264enc)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(x264enc, x264enc_caps_filter)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(x264enc_caps_filter, rtph264pay)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(rtph264pay, rtph264pay_caps_filter)</div><div class=""> <a href="http://gst.element.link/" class="">Gst.Element.link</a>(rtph264pay_caps_filter, <span style="color: #7f0055;font-weight: bold;" class="">self</span>.webrtcbin)</div><div class=""> <span style="color: #7f0055;font-weight: bold;" class="">self</span>.pipeline.set_state(Gst.State.PLAYING)</div><br class=""></div></div></div></div></div></blockquote></div><br class=""></div></body></html>