[gst-devel] Concepts: GStreamer device selection and beyond (audio/video)

[Martin Szulecki]

Hello to this mailing list first of all. :)


--- Introduction: Device Selection Handling

I have been doing some work on gstreamer-properties to integrate easy
device selection for the audio/video src/sink element setup.

It is vital for some users to be able to (easily as for User Joe) select
a specific device to be used in the "multimedia system" in cases like:

- Switch from soundcard output to USB headset
- Video input device (Webcam instead of TV decoder for instance) as
global preference as well as possibly in an GStreamer enabled
application.
...

As the advanced user certainly is able to squeeze some device property
values to use from somewhere in their custom pipelines, it should
certainly be as simple as selecting a device in a list.

A couple of bugs have been filed in gnome-media, control-center and
gstreamer which somehow arise due to people stumbling among the same
end.

Some solutions implement their own device probing, some use HAL to
detect ALSA devices, others (like I thought would be best) use the
PropertyProbe interface of some GST elements (alsa, v4l) to get a list
of devices but there is a lack of consistency and unification among the
desktop.

Therefore I was advised to post to this mailing list to discuss how
devices (and beyond, e.g. further device capabilities) should be handled
in order for maximum efficiency, modular design and integration into
GStreamer and applications using it.


--- Device Handling 

Let's look at the current way of how things are done, different
approaches and possible further ideas on a generic way of doing stuff
(time for you contributing ideas).

# Current:

1. Some audio/video elements in GStreamer (alsa, v4l, v4l2) have a
unspecified device (r/w) and device-name (read only) property
2. To get a list you create an element, use the PropertyProbe interface
to get a list of values for the "device" property, set it to something
below READY state and read the "device-name" property which should
contain a human readable name
3. Populate your drop-down with this information

The negative stuff here is that it probes the device from the backend
for each pass which can cause issues. Caching was partly added but
yields wrong in times of pluggable devices so the list is always probed
now.

Errors can occur during the probing, it adds a lack of user experience
and possibly delays to the selection system.

The biggest argument against the current way I guess is that it actually
requires a quite complicated bunch of code to actually do a rather
simple thing: Get a list of devices and select upon.

# Other options and thoughts:

- Latest gnome-sound-properties allows sound device selection using a
mix of HAL/gconf (halaudiosink/src) for ESD
- GST element device information specific API additions could make it
easy to implement functionality for element plugins
- HAL offers a (standardized) way to check for devices without requiring
to probe, maybe make the "device" property an "UDI" in general is
easier?
- "Outsource" the problem (remove device probing code from GST elements
completely and put in an external utility which uses HAL)
- Stick to current way, just add more PropertyProbe interface support to
other elements (oss, esd etc.) and make gstreamer-properties use it
exclusively
- Establish hal[audio|video(|mixer?)][sink|src] as standard and make it
auto-select the required backend specific element for a chosen device.
Make it have a device list property sourced from HAL with human readable
name and UDI which also avoids probing (and thus blocking).
gstreamer-properties would save HAL preference in gconf. Applications
could use the HAL API to get devices and the hal elements to set devices
in their pipelines.
...

I personally think the last option might be the way to go. As everything
would be UDI-based this would aswell allow for an utility (or extension
of gstreamer-properties) which could provide easy configuration of the
selected device's advanced options in some distant future. (Image
settings, soundcard advanced options...).


--- Goals:

- Ability to easily get a list of devices in a generic desktop friendly
way for audio and video systems
- Make an element use a specific device easily
- Save global device preferences at a common location
- Allow per-application device selection while using the global
preferences initially (Videoconferencing might want to use the headset
instead of the soundcard, Sample editor might want to use a different
soundcard or microphone from an headset)
- Make it painless on the user side to select devices (and maybe their
configuration) for their GStreamer enabled desktop
- Make it easy to implement code to offer device selection for
developers within their GStreamer enabled applications


--- Notes:

Also mind advanced device specific properties! (Multi-channel
soundcards, sub-devices, image setup on specific video devices etc.)

Might this be just a general question of moving GST closer towards HAL
integration?


Looking forward to your feedback and hopefully a good concept as the
result.


--- Martin Szulecki