FsAppTransmitter / dummy call in nullprpl

David Woodhouse dwmw2 at infradead.org
Fri Feb 2 13:31:15 UTC 2018


On Mon, 2017-12-11 at 10:13 +0000, David Woodhouse wrote:

> On Sun, 2017-12-10 at 21:02 +0000, David Woodhouse wrote:
> > I'm thinking of following the precedent set by FsShmTransmitter and
> > abusing the host and username fields of the FsCandidate, except not
> > with filenames as FsShmTransmitter does it, but with pipeline
> > descriptions to be handled by gst_parse_launch().
> 
> OK, so that much works, but I can't used named elements; those have to
> have been created in the *same* pipeline description. So this code
> doesn't work because there's no way to make it find "MyTestSrc" AFAICT:

So I got this working. I can create the named element in the
FsAppTransmitter pipeline, then I can find it again from the outside
with gst_bin_get_by_name().

My own protocol code can now feed Opus frames up from the GstAppSrc,
and receive them from the GstAppSink, and I can actually have a
conversation. Yay!

However... I need to do a bunch of stuff for myself. There are
timestamp fields in the wire protocol, which I need to manually keep in
sync with the timestamps on the GstBuffers, and I also need a jitter
buffer.

Now that now my life isn't *entirely* dominated by CPU vulnerability
work I'm starting to take a look again. It occurred to me that I'd
actually be better off faking RTP, and just copying my protocol's
timestamps into the corresponding field of an RTP packet. That way I
can avoid reinventing the wheel.

Before I do the step of faking RTP myself, I started with purely adding
rtpopuspay/rtpopusdepay in my pipelines and switching to
fsrtpconference. And now I'm back to "I am clueless and I don't know
why this doesn't work mode" and asking for help again... :)

I'm still working on getting permission to actually publish this PRPL
under LGPL, but below is a pseudo-patch which shows what was working,
and what I've done. I can probably do this in the dummy PRPL as I did
before, if anyone really wants to be able to test it locally.

The GStreamer pipeline is at http://david.woodhou.se/conf.svg

When I press a key it does correctly send DTMF. But although the audio
from the microphone *looks* like it's hooked up, nothing is arriving.
And the incoming audio doesn't seem to be hooked up at all. And I never
see 'farstream-recv-codec-changed'.

Any further clues would be, as before, most gratefully received.

 static void on_audio_state(ChimeCall *call, ChimeAudioState audio_state, struct chime_chat *chat)
 {
        purple_debug(PURPLE_DEBUG_INFO, "chime", "Audio state %d\n", audio_state);
 
        const gchar *name = chime_call_get_alert_body(chat->call);
 
        if (audio_state == CHIME_AUDIO_STATE_AUDIO_MUTED && chat->media) {
                purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_MUTE, "chime", name, FALSE);
        } else if (audio_state == CHIME_AUDIO_STATE_AUDIO && chat->media) {
                purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_UNMUTE, "chime", name, FALSE);
        } else if (audio_state == CHIME_AUDIO_STATE_AUDIO && !chat->media) {
                PurpleMediaManager *mgr = purple_media_manager_get();
                chat->media = purple_media_manager_create_media(purple_media_manager_get(),
                                                                chat->conv->account,
-                                                               "fsrawconference",
+                                                               "fsrtpconference",
                                                                name,
                                                                TRUE);
                if (chat->media) {
-                       const gchar *caps = "audio/x-raw,format=(string)S16LE,layout=(string)interleaved,channels=(int)1";
+                       const gchar *caps = "application/x-rtp,media=(string)audio,payload=(int)96,encoding-name=(string)OPUS";
 
                        gboolean r = purple_media_add_stream(chat->media, "chime", name,
                                                             PURPLE_MEDIA_AUDIO, TRUE,
                                                             "app", 0, NULL);
                        gchar *srcname = g_strdup_printf("chime_src_%p", call);
                        gchar *sinkname = g_strdup_printf("chime_sink_%p", call);
                        /* Without the capsfilter (even though there's another capsfilter almost immediately after it in the pipeline, it doesn't work */
-                       gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! opusdec ! capsfilter caps=%s",
-                                                        srcname, caps);
-                       gchar *sinkpipe = g_strdup_printf("opusenc bitrate=16000 bitrate-type=vbr ! appsink name=%s async=false", sinkname);
+                       gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! rtpopuspay ! capsfilter caps=%s", srcname, caps);
+                       gchar *sinkpipe = g_strdup_printf("rtpopusdepay ! appsink name=%s async=false", sinkname);
 
                        PurpleMediaCandidate *cand =
                                purple_media_candidate_new(NULL, 1,
                                                           PURPLE_MEDIA_CANDIDATE_TYPE_HOST,
                                                           PURPLE_MEDIA_NETWORK_PROTOCOL_UDP,
                                                           sinkpipe, 0);
                        g_object_set(cand, "username", srcpipe, NULL);
                        g_free(sinkpipe);
                        g_free(srcpipe);
 
                        GList *cands = g_list_append (NULL, cand);
                        GList *codecs = g_list_append(NULL,
-                                                     purple_media_codec_new(1, caps, PURPLE_MEDIA_AUDIO, 0));
+                                                     purple_media_codec_new(96, "OPUS", PURPLE_MEDIA_AUDIO, 0));
+//                      g_object_set(codecs->data, "channels", 1, NULL);
+                       purple_media_codec_add_optional_parameter(codecs->data, "farstream-recv-profile", "rtpopusdepay ! opusdec");
+                       purple_media_codec_add_optional_parameter(codecs->data, "farstream-send-profile", "opusenc ! rtpopuspay");
+//                      purple_media_codec_add_optional_parameter(codecs->data, "sprop-stereo", "0");
+//                      purple_media_codec_add_optional_parameter(codecs->data, "stereo", "0");
+                       purple_media_get_local_candidates(chat->media, "chime", name);
                        purple_media_add_remote_candidates(chat->media, "chime", name, cands);
+                       purple_media_set_send_codec(chat->media, "chime", codecs->data);
                        purple_media_set_remote_codecs(chat->media, "chime", name, codecs);
 
                        GstElement *pipeline = purple_media_manager_get_pipeline(mgr);
                        GstElement *appsrc = gst_bin_get_by_name(GST_BIN(pipeline), srcname);
                        GstElement *appsink = gst_bin_get_by_name(GST_BIN(pipeline), sinkname);
                        g_free(srcname);
                        g_free(sinkname);
 
                        gst_app_src_set_size(GST_APP_SRC(appsrc), -1);
                        gst_app_src_set_max_bytes(GST_APP_SRC(appsrc), 100);
                        gst_app_src_set_stream_type(GST_APP_SRC(appsrc), GST_APP_STREAM_TYPE_STREAM);
 //                     gst_base_src_set_live(GST_BASE_SRC(appsrc), TRUE);

                        chime_call_install_gst_app_callbacks(chat->call, GST_APP_SRC(appsrc), GST_APP_SINK(appsink));
                        g_signal_connect(chat->media, "state-changed", G_CALLBACK(call_media_changed), chat);
                        g_signal_connect(chat->media, "stream-info", G_CALLBACK(call_stream_info), chat);
 
                        purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_ACCEPT, "chime", name, FALSE);
                        GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(purple_media_manager_get_pipeline(mgr)), GST_DEBUG_GRAPH_SHOW_ALL, "chime conference graph");
                }
        }
 
 
 }
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5213 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/farstream-devel/attachments/20180202/00a5b218/attachment.bin>


More information about the Farstream-devel mailing list