Multimedia support (Re: X Developers' Summit)

[Carsten Haitzler (The Rasterman)]

On Mon, 16 Jul 2007 16:55:46 +0200 Helge Bahmann <hcb at chaoticmind.net> babbled:

> Am Montag, 16. Juli 2007 15:25 schrieb Nicolas Mailhot:
> > You probably need to look hard at pulseaudio. Colin was too polite to
> > write it, but major Linux distributions are planning a migration to
> > pulseaudio. They've been burnt so deeply by previous audio system
> > reimplementations/duplications their reaction to yet another audio
> > project is likely to be overwhelmingly negative.
> 
> I understand that, but I am less interested in what distributions plan to do 
> for pragmatic reasons short-term than what would technically be the best 
> approach long-term. As for interfaces, it should be trivial to emulate the 
> PulseAudio API on top of X-based audio. It is certainly possible to provide 
> an ALSA interface, which should cover the overwhelming majority of 
> applications already.
> 
> I remind you that my goal is not to provide just "network audio", but 
> instead "perfectly synchronized video/audio" and fine-grained application 
> control over the whole presentation. This is incredibly difficult if you are 
> building your audio system "A" besides the graphics system "X" -- for 
> synchronisation purposes you need at the very least to be able to refer to 
> events in "A" from "X" (and vice versa); the interaction between "A" and "X" 
> becomes messier the more fine-grained control you give to applications.
> 
> The point is you can either
> - write a separate audio daemon in ~60000 lines of code and still be unable
> to synchronize video and audio usefully (mas, PulseAudio)
> - write an X audio extension in ~4000 lines of server code utilizing the 
> infrastructure that is in place and get perfect synchronisation for free 
> (okay, admittedly this is another 1000 LoC)
> - (or ignore X altogether and roll your own presentation system as NMM does)
> 
> I think there are compelling reasons for taking approach number two, and I
> simply want to demonstrate that and how it can be done. Sorry if I am late to 
> the party, but sometimes ideas take time :)
> 
> > (Also going through xorg will remind everyone of the mas debacle.)
> 
> yes, and frankly I fail to see what PulseAudio is doing fundamentally 
> different from mas when it comes to synchronisation. Not that it is 
> impossible, but it will get very very messy, and I can show that integrating 
> it into X to begin with neatly solves this problem.

personally - i think going via x proto has merit. you have a point. the only
problem now is that it becomes x's job to syncronise audio within itself (and
you thought you got rid of the problem!). you basically shifted the problem
from the app to x. that's fine - but it's not a silver bullet. you will
probably now need to content with mixing audio in client-space (for multiple
audio streams), mixing buffers (or audio stream priorities to block off audio
from other clients), need to handle audio device buffer sizes for sync (and
when you add a mixing buffer this really get a bit nasty when combined with
needing to run basically realtime as skips in audio when you run out of enough
cpu to do the mixing really sound horrible - dropping frames is almost heavenly
bliss in comparison).

anyway - point is - audio is its own can of worms. moving it to x is good from
one point - if it IS a remote display the DISPLAY itself will play the audio-
i.e. - the audio is where it should be - the user. as a user i have no desire
for my audio to play out of the server in the room 5 miles away. i want it here
at my terminal where i am, and an x-based audio layer would make that simple
and easy to achieve. for local clients this is just "yet another route to the
audio device" (nb - you will want to be considering shm transports for local
clients).

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster at rasterman.com
裸好多
Tokyo, Japan (東京 日本)