request for comments: plan for audio/video policy integration in gstreamer

[Wim Taymans]

On 10/12/2012 03:44 PM, Alexander Kanavin wrote:
> Hello,
>
> me and a few colleagues at Intel Open Source Technology Center are
> working on adding multimedia policy mechanisms to the open source stack
> so that it could be used by the automotive industry in IVI (in-vehicle
> infotainment) systems. In particular, I've been looking at how to
> integrate policy into gstreamer, and the following is the direction we'd
> like to take:
>
> 1. On IVI systems there is a policy daemon which keeps track of the
> system state, has a database of policy rules, and makes decisions about
> any ongoing multimedia playback:
>
> a) whether any particular stream should be paused or can continue to
> play (think of a phone call interrupting music playback)
> b) where any particular stream should be routed (for example front seat
> or back seat)
>
> 2. All of the above should be hidden from applications; they should not
> be concerned with policy or routing and can use gstreamer as they always
> did for playback.
>
> 3. Policy integration in gstreamer is handled by a policy element. All
> implementations of that element subclass from GstPolicy (which is a
> subclass of GstElement and brings together things common to all policy
> elements). One such common thing to all policy implementations is a
> stream 'role' property that applications can set if they want to, for
> example 'phonecall' or 'music' (this property affects policy decisions).
> The property would be picked up from the policy element and exposed to
> applications by playbin.
>
> 4.  There is also an autopolicy element that wraps a
> distribution-specific policy element if it can find one, or otherwise a
> default policy element that does nothing. Autopolicy element could be
> located for example in playsink bin just after stream synchronizer
> element. Since policy needs to be enforced, there should be no way to
> replace it by the applications.
>
> 5. The policy element that we plan to implement for IVI use cases would:
>
> a) handle all communication with the policy daemon
> b) receive ongoing information from policy daemon about routing and
> whether the stream should be paused or resumed, based on that 'role'
> property and whether there is audio, video or both in the stream.
> c) routing information is passed downstream as GstMeta. Sinks can
> cherry-pick and propagate this information to audio and video subsystems.
> d) policy element is using gst_element_get_parent() to find out the
> top-level pipeline element, and will issue state change requests
> (pause/resume) on it as instructed by policy daemon.
>
> We plan to make this work fully open source and hope to have it
> upstream, but first of course we'd like to know what you think about it.
> Please let us know.

Why no use pulseaudio? I believe it can do all this already. It doesn't
sound like something that GStreamer should do.

Also, you can't pause the pipeline from an element. We have a REQUEST_STATE
message for this purpose (which pulseaudio sends to the pipeline when a
policy requires you to pause a stream).

Wim

>
>
> Thanks,
> Alex
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel