[Accessibility] RE: Multimedia framework requirements

[Pete Brunet]

Marc, How does the need for instant echoing of keyed characters when 
entering text fit in with this situation?  Thanks, Pete

=====
Pete Brunet, (512) 838-4594, TL 678-4594, brunet at us.ibm.com, ws4g
IBM Accessibility Architecture and Development, 11501 Burnet Road, MS 
9026D020, Austin, TX  78758

----------------------------------------------------------------------
Date: Sat, 5 Mar 2005 17:55:27 -0700
From: "Marc Mulcahy" <marc at plbb.net>
Subject: RE: [Accessibility] Multimedia framework requirements
To: "Gary Cramblitt" <garycramblitt at comcast.net>,
                 <accessibility at lists.freedesktop.org>
Message-ID: <KKEGJCDELINGIGICHANAGEKBEDAA.marc at plbb.net>
Content-Type: text/plain;                charset="iso-8859-2"

Well, for what it's worth, here is my $.02.

1. We in the accessibility community will never succeed in trying to
re-invent the multimedia server.  There have been many attempts by people
with expertise in multimedia with varying degrees of success.  So I think
the right approach is to focus on selecting an existing solution that 
comes
closest to what we need, and either living with it, or proposing changes
which will bring it closer to what we need.

2. The biggest oversight in gnome-speech was that it did not directly 
handle
the audio coming out of software synthesizers.  Given my experience with
several commercial and open source speech engines, I came to the 
conclusion
that the speech framework *must* have control over the audio samples and
where they go.  If we leave it up to the speech engines, they will all
implement things differently, and we have much less of a good chance of
providing a good experience for the end user.  Having control over the 
audio
gives us better control over quick startup and stop times, as well as the
ability to route speech to diferent destinations-- files, headsets,
speakers, telephone lines, etc.

3. To my mind, ALSA comes the closest to what we need in an audio 
framework
on Linux.  It's now standard, and provides methods for mixing audio 
streams
on soundcards which can't do it in hardware.  The prioritization of 
audio--
I.E., muting the MP3 player when the computer needs to speak something or
when a user receives an internet phone call, is the only piece which 
appears
to be missing.

Another audio-related aside...  I think there's been some
mischaracterization of a requirement.  Everyone seems to suggest that what
we need is low-latency in an audio server or environment, and I'm not
convinced that this is the case.  You need low-latency, or at least good
synchronization, if for example you want to animate a character using
text-to-speech as the voice.  But, I think from an accessibility point of
view, what we really need is quick start and shut up times, not 
necessarily
low latency, although low latency is better.  For example, from a blind
usability point of view, I don't care if the ap sends the sound card a 128
KB buffer of audio or a 1 KB buffer of audio, as long as the sound stops
immediately when I press a key, and as long as it starts immediately when
there's something to be spoken.

My experience shows that low-latency is in fact not necessarily desirable
when working with speech.  Presumably speech is a background process which
goes on while other more intensive tasks are happening in the foreground--
copying a file, filtering audio, or something of that sort.  the lower the
latency, the harder it is to keep speech happy in the background, 
especially
during periods of high disk activity or network load.

Rather than having to feed the soundcard 1 K blocks of data, I'd rather
synthesize 64 K of data, and dump it to the sound card, and let the DMA
controller transfer it while the processor does something else.  and as 
long
as I can shut it up immediately, the user doesn't know the difference.

Marc

-----Original Message-----
From: accessibility-bounces at lists.freedesktop.org
[mailto:accessibility-bounces at lists.freedesktop.org]On Behalf Of Gary
Cramblitt
Sent: Saturday, March 05, 2005 6:20 AM
To: accessibility at lists.freedesktop.org
Subject: Re: [Accessibility] Multimedia framework requirements


On Friday 04 March 2005 03:43 pm, Hynek Hanke wrote:
> 2 Audio requirements

You may want to think about supported audio formats.  Most existing synths
seem to produce .wav files (Microsoft riff) or .au.

Also, there's the issue of how to deliver the audio to the audio 
framework.
Streams could be more efficient than files?  The TTS API discussion has 
this
as an unresolved item.

--
Gary Cramblitt (aka PhantomsDad)
KDE Text-to-Speech Maintainer
http://accessibility.kde.org/developer/kttsd/index.php

------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freedesktop.org/archives/accessibility/attachments/20050307/f5297089/attachment.htm