hi, I am pleased to introduce GSmartMix. GSmartMix has recently been accepted to Google SoC. It aims is to bring a new sound experience in Gnome. Thus, I guess this effort should be discussed here.  Ensonic, Stefan Kost, is the mentor of this project. Raphel Slinckx is the co-mentor. If you are not already familiar with this project, you could have a look at <a href="http://live.gnome.org/GSmartMix" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://live.gnome.org/GSmartMix </a> before reading this.  I will also do my best to change the GNOME sound support, but *it's not* the main goal. For the moment, GSmartMix is a project that I want to design to be run aside and optionnaly, without any API change. The following are some technical suggestions on GSmartMix, I would like to discuss with you. Please note: GNOME API and esd discussions will go in gnome.multimedia mailing list. I don't make any difference between rule or class for the moment: it's a description of the sound: "music", "voice", "event"... === UI === The ui provide a per class sound level sliders, and some rule parameters: - foreground sound level - background sound level - fading speed - and list of class that we put in foreground when this class has to play (queue policy: first in stay in fg (fifo), last in go in fg (lifo), none (all fg))  I would like to add a new tab in the GNOME "Volume Control" application with this content: <a href="http://live.gnome.org/GSmartMix/TechnicalChoices?action=AttachFile" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://live.gnome.org/GSmartMix/TechnicalChoices?action=AttachFile </a> Regarding the Topaz mockup of volume controler and the per-app volume control (<a href="http://static.flickr.com/20/70003494_668cfdc0dd.jpg" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://static.flickr.com/20/70003494_668cfdc0dd.jpg</a>): GSmartMix benefits will partially fit this. I don't know how we could deal with "multiple instance of an application" problem... This is why I was briefly thinking of a Metacity integration, in order to differenciate each sound source easily. But I am wondering if one "foreground app" per class for common sounds (voice, video, music..) will help to clean up this list. This way, only "foreground app" will be shown in the list. Only some class, like "events" won't need this "front" scheme. We don't care of a per app "event" sound level probably.Is it really what we want? === Sinks: a new element or a gstbaseaudiosink integration === I have tried to depict my initial overview of GSmartMix here: <a href="http://live.gnome.org/GSmartMix/TechnicalChoices?action=AttachFile&do=view&target=overview.png" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://live.gnome.org/GSmartMix/TechnicalChoices?action=AttachFile&do=view&target=overview.png</a>   Different design solutions can be thought up. What will be the best or the consensus? ==== "Base patch" solution ==== The idea is to handle dbus communication at the baseaudiosink class level. ensonic, how would be handled the volume control/transformation at this level? This way, every audiosink will be seen over dbus. Drawback: - all compiled audio sinks will have a libdbus dependency (if we compile gstlib with this support) - intrusive patch in gst - every app will have to suffer/benefit the smart level control ==== "New element" solution ==== Formerly, I was thinking of a kind of GStreamer "identity" bin elements, inserted before the real audio sink. They will catch some gstramer events (status/streaminfo..) and send them over dbus. After that they will obey GSmartMix server orders and update a volume level. The "volume" element manipulated would be found by looking up the parent bin. The "volume_element" property could be overriden. If none is found, a new one will be added in GSmartSink bin in order to control volume output. This way, I hope to be able to provide GSmartMix without having to change anything in the application side and in GStreamer. Look at this sketch: <a href="http://photos1.blogger.com/blogger/2015/2884/1600/sec.2.png" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://photos1.blogger.com/blogger/2015/2884/1600/sec.2.png </a> Drawback: - another plugin => a different pipeline - all applications/usage will not benefit from GSmartMix (those who use directly alsasink for instance) - more complex solution? ==== Polypaudio ==== jeff pointed me out Polypaudio. I was not thinking hard of using it, because it sounded like an odd but good esd replacement. Which is not the aim of my work. But, I must admit I am quite impressed by its state. It features some of the required interfaces for my project. Definetly, my will is to fill the gab between gnome, gstreamer and polypaudio. Recent discussion with lennart on #polypaudio about GSmartMix: <a href="http://pastebin.com/747392" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://pastebin.com/747392</a> Drawback (if the effort is to patch polypaudio): - the volume level adjusted at the server side will not be updated in the client side (for example, the volume slider of your favourite music player): I believe I can fix this or is there already a solution in polypaudio? The polypaudio author, Lennart, will post on this subject, onthe  GStreamer GstMixer flaw. - we still depends officialy on a sound server (and not on ALSA/dmix), which could be badly interpreted ==== Remarks ==== - I believe sinks must initialise and "block" sound output until the dbus server did not reply to adjust their level. - of course, I would also like to bring new things to the client side, like the ability to describe sound, like the ability to give sound "class" and "priority". Could it be done with standard GStramer API, with a bus message ? === GSmartMix, a dbus server === I like the simplicity of devil's pie. I wonder if I could improve s-exp interpreter to support GSmartMix needs (fifo on class, loops on other sinks ?).  So that the server could be written with light C code. If more freedom is required, Python is probably the language of choice. But, it could be written differently. I also wonder how the dbus server could monitor its slaves: the communication need to be in a "connected" mode with the peer (I think of an application crash, an abnormal behaviour...). -- Marc-André Lureau, GSmartMix