<div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le dim. 22 août 2021 07 h 45, Neil Young via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Finally giving up on this. Python, webrtcbin and H.264 offer - this does not work</div><div style="word-wrap:break-word;line-break:after-white-space" dir="auto"></div></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Hi Neil, I'm sorry this did not work you and no one was available in your required delay to freely support you.</div><div dir="auto"><br></div><div dir="auto">This answer is simply to state that this works for others, the webrtcbin and rtpbin pipeline are known to be difficult, but not broken (except for few missing webrtc features of course).</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" dir="auto"><br><div dir="auto"><br></div><div dir="auto"><br><div><br><blockquote type="cite"><div>Am 21.08.2021 um 20:23 schrieb Neil Young <<a href="mailto:foreverneilyoung@googlemail.com" target="_blank" rel="noreferrer">foreverneilyoung@googlemail.com</a>>:</div><br><div><div style="word-wrap:break-word;line-break:after-white-space">I'm hanging on this for a week now, and it seems so easy, but whatever I do, I can't make it work<div><br></div><div>My app is a Python GST app, running on 18.4. It supports two modes: "call" and "answer". In call mode the app immediately issues an H.264 or VP8 OFFER (depending on a setting). This works in 100 % of the cases. </div><div><br></div><div>What also works is when the app ANSWERs a VP8 offer. Just H.264 OFFERS are not working, even though the SDP answer looks good. I just don't get the video to remote.</div><div><br></div><div>My app works like so:</div><div><br></div><div>1) When the app has a connection to the signaling server it creates a webrtcbin object:</div><div><br></div><div><div style="background-color:rgb(255,255,255);font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;line-height:21px;white-space:pre-wrap"><div>    <span style="color:#210de5;font-weight:bold">def</span> <span style="color:#1200bb">build_webrtcbin</span>(<span style="color:#ab3c11;font-style:italic">self</span>):</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"webrtcbin"</span>)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.set_property(<span style="color:#ba2500">"bundle-policy"</span>, <span style="color:#ba2500">"max-bundle"</span>)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.set_property(<span style="color:#ba2500">"stun-server"</span>, <span style="color:#ba2500">"<a rel="noreferrer">stun://stun.l.google.com:19302</a>"</span>)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'on-negotiation-needed'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_negotiation_needed)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'on-ice-candidate'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_ice_candidate)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'pad-added'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_pad_added)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'notify::connection-state'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_connection_state)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'notify::signaling-state'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_signaling_state)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'notify::ice-connection-state'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_ice_connection_state)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.connect(<span style="color:#d126d1">'notify::ice-gathering-state'</span>, <span style="color:#7f0055;font-weight:bold">self</span>.on_ice_gathering_state)</div><br><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline <span style="color:#7f0055">=</span> Gst.Pipeline.new()</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(<span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin)</div><div><br></div><div>Nothing special I guess.</div><br></div></div><div style="background-color:rgb(255,255,255);line-height:21px"><div style="font-family:Verdana;white-space:normal">2) When the SDP OFFER comes in, the app does this:</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">2a) It starts the H.264 pipeline with `start_pipeline`. </div><div style="font-family:Verdana;white-space:normal">2b) It emits the remote description to the local webrtcbin</div><div><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;white-space:pre-wrap;line-height:21px"><span style="color:#7f0055;font-weight:bold"><br></span></div><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;white-space:pre-wrap;line-height:21px"><span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.emit(<span style="color:#d126d1">'set-remote-description'</span>, desc, promise)</div><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;white-space:pre-wrap;line-height:21px"><br></div><div style="line-height:21px">2c) It creates an answer calling `self.create_answer()`</div><div style="line-height:21px"><br></div><div style="line-height:21px"><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;line-height:21px;white-space:pre-wrap"><div>    <span style="color:#210de5;font-weight:bold">def</span> <span style="color:#1200bb">create_answer</span>(<span style="color:#ab3c11;font-style:italic">self</span>):</div><div>        promise <span style="color:#7f0055">=</span> Gst.Promise.new_with_change_func(<span style="color:#7f0055;font-weight:bold">self</span>.on_answer_created, <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin, <span style="color:#0a9dff">None</span>)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.emit(<span style="color:#d126d1">'create-answer'</span>, <span style="color:#0a9dff">None</span>, promise)</div><div style="line-height:21px"><div>    </div><div><span style="color:#210de5;font-weight:bold">    def</span> <span style="color:#1200bb">on_answer_created</span>(<span style="color:#ab3c11;font-style:italic">self</span>, <span style="color:#ab3c11;font-style:italic">promise</span>, <span style="color:#ab3c11;font-style:italic">_</span>, <span style="color:#ab3c11;font-style:italic">__</span>):</div><div>        <span style="color:#d126d1">''' Local WebRTC stack has created an answer '''</span></div><div>        promise.wait()</div><div>        reply <span style="color:#7f0055">=</span> promise.get_reply()</div><div>        answer <span style="color:#7f0055">=</span> reply.get_value(<span style="color:#d126d1">'answer'</span>)</div><div>        promise <span style="color:rgb(127,0,85)">=</span> Gst.Promise.new()</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin.emit(<span style="color:#d126d1">'set-local-description'</span>, answer, promise)</div><div>        promise.interrupt()</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.send_sdp_answer(answer) # Forwarding the answer to remote using the signaling server</div></div></div></div><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;white-space:pre-wrap;line-height:21px"><br></div></div><div style="font-family:Verdana;white-space:normal">Observations so far:</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">- It doesn't work if I don't set a payload-type which matches a payload type carried by the OFFERER or if I omit the payload-type. With ALL browsers it is sufficient to use the payload-type carried by the first H.264 element in the OFFER</div><div style="font-family:Verdana;white-space:normal">- It is not necessary to set the `profile-level-id`</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">The best result I can achieve with this flow is a spinning wheel in the browser. No video.</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">BUT: </div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">When I issue a call to `start_pipeline` somewhere shortly after the start (I use the moment, when I have a connection with the signaling server for this) with _exactly_the_same_payload_type_that_will_match_later_, I 100% have a video...</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">This MUST be a side effect, because the log complains about the second `start_pipeline` (warnings, it seems to ignore it).... But it works. Of course this is not a working solution, because</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">a) I would have to know the payload-type beforehand (it doesn't work if I use another payload-type in the first start)</div><div style="font-family:Verdana;white-space:normal">b) I would have to parse the incoming SDP for the first H.264 (which is doable)</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal">What am I doing wrong?</div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal"><br></div><div style="font-family:Verdana;white-space:normal"><div style="font-family:"Fira Code",Menlo,Monaco,"Courier New",monospace,Menlo,Monaco,"Courier New",monospace;line-height:21px;white-space:pre-wrap"><div>    <span style="color:#210de5;font-weight:bold">def</span> <span style="color:#1200bb">start_pipeline</span>(<span style="color:#ab3c11;font-style:italic">self</span>, <span style="color:#ab3c11;font-style:italic">payload_type</span>):</div><div>        videotestsrc <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"videotestsrc"</span>, <span style="color:#ba2500">"videotestsrc"</span>)</div><div>        videotestsrc.set_property(<span style="color:#ba2500">"is-live"</span>, <span style="color:#0a9dff">True</span>)</div><div>        videotestsrc.set_property(<span style="color:#ba2500">"pattern"</span>, <span style="color:#ba2500">"smpte"</span>)</div><br><div>        videotestsrc_caps <span style="color:#7f0055">=</span> Gst.caps_from_string(<span style="color:#ba2500">"video/x-raw, width=1280, height=720"</span>)</div><div>      </div><div>        videotestsrc_caps_filter <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"capsfilter"</span>)</div><div>        videotestsrc_caps_filter.set_property(<span style="color:#ba2500">"caps"</span>, videotestsrc_caps)</div><br><div>        videoconvert <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"videoconvert"</span>, <span style="color:#ba2500">"videoconvert"</span>)</div><br><div>        x264enc <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"x264enc"</span>, <span style="color:#ba2500">"x264enc"</span>)</div><div>        x264enc.set_property(<span style="color:#ba2500">"threads"</span>, <span style="color:#ff8000">4</span>)</div><div>    </div><div>        x264enc_caps <span style="color:#7f0055">=</span> Gst.caps_from_string(<span style="color:#ba2500">"video/x-h264, profile=constrained-baseline"</span>)</div><br><div>        x264enc_caps_filter <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"capsfilter"</span>)</div><div>        x264enc_caps_filter.set_property(<span style="color:#ba2500">"caps"</span>, x264enc_caps)</div><br><div>        rtph264pay <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"rtph264pay"</span>, <span style="color:#ba2500">"rtph264pay"</span>)</div><div>        rtph264pay.set_property(<span style="color:#ba2500">"config-interval"</span>, <span style="color:#098eff;font-weight:bold">-</span><span style="color:#ff8000">1</span>)</div><div>        rtph264pay.set_property(<span style="color:#ba2500">"aggregate-mode"</span>, <span style="color:#ba2500">"zero-latency"</span>)</div><br><div>        rtph264pay_caps <span style="color:#7f0055">=</span> Gst.caps_from_string(<span style="color:#ba2500">"application/x-rtp, media=video, encoding-name=H264"</span>)</div><div>        rtph264pay_caps.set_value(<span style="color:#ba2500">"payload"</span>, payload_type)</div><br><div>        rtph264pay_caps_filter <span style="color:#7f0055">=</span> Gst.ElementFactory.make(<span style="color:#ba2500">"capsfilter"</span>)</div><div>        rtph264pay_caps_filter.set_property(<span style="color:#ba2500">"caps"</span>, rtph264pay_caps)</div><br><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(videotestsrc)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(videotestsrc_caps_filter)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(videoconvert)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(x264enc)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(x264enc_caps_filter)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(rtph264pay)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.add(rtph264pay_caps_filter)</div><br><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(videotestsrc, videotestsrc_caps_filter)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(videotestsrc_caps_filter, videoconvert)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(videoconvert, x264enc)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(x264enc, x264enc_caps_filter)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(x264enc_caps_filter, rtph264pay)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(rtph264pay, rtph264pay_caps_filter)</div><div>        <a href="http://gst.element.link/" target="_blank" rel="noreferrer">Gst.Element.link</a>(rtph264pay_caps_filter, <span style="color:#7f0055;font-weight:bold">self</span>.webrtcbin)</div><div>        <span style="color:#7f0055;font-weight:bold">self</span>.pipeline.set_state(Gst.State.PLAYING)</div><br></div></div></div></div></div></blockquote></div><br></div></div></blockquote></div></div></div>