Echo acoustic on iOS device

[Elio Francesconi]

Hi Sebastian,
	I’ve modified osxaudio plugin to use Voice-Processing I/O unit instead of Remote I/O unit. I think you should add it to GStreamer project to support also VoIP or voice-chat application.
To use Voice-rocessing I/O unit I did these changes:
> in gstosxcoreaudioremoteio.c 
> 
> static gboolean
> gst_core_audio_open_impl (GstCoreAudio * core_audio)
> {
>   return gst_core_audio_open_device (core_audio, kAudioUnitSubType_VoiceProcessingIO,
>                                        "VoiceProcessingIO");
>   //return gst_core_audio_open_device (core_audio, kAudioUnitSubType_RemoteIO,
>   //    "RemoteIO");
> }
This patch was not enough to solve the issue because I received status = -50 after some calls of AudioUnitRender in getosxaudiosrc.c, it seems AudioUnitRender modifies  buf->core_audio->recBufferList->mBuffers[0].mDataByteSize based on data returned, so before call AudioUnitRender it requires to set mDataByteSize today I’ve created mine osxaudio plugin with these changes, hardcoding "bytes per frame” and “channels” and solved my problem, I did’d check the buffer allocated because it’s big enough.
buf->core_audio->recBufferList->mNumberBuffers=1;
buf->core_audio->recBufferList->mBuffers[0].mDataByteSize=inNumberFrames*4/*where 4 is bytes per frame*/;
buf->core_audio->recBufferList->mBuffers[0].mNumberChannels=1 /*num of channels*/;
status = AudioUnitRender (buf->core_audio->audiounit, ioActionFlags,
      inTimeStamp, inBusNumber, inNumberFrames, buf->core_audio->recBufferList);

  if (status) {
    GST_WARNING_OBJECT (buf, "AudioUnitRender returned %d", (int) status);
    return status;
  }

Have a nice day
Elio

On 05 Jul 2014, at 12:09, Elio Francesconi <elio.francesconi at gmail.com> wrote:

> Hi Sebastian,
> reading documentation I’ve found the issue, 
> 
> https://developer.apple.com/library/ios/documentation/MusicAudio/Conceptual/AudioUnitHostingGuide_iOS/AudioUnitHostingFundamentals/AudioUnitHostingFundamentals.html
> I/O Units
> iOS provides three I/O units. The Remote I/O unit is the most commonly used. It connects to input and output audio hardware and gives you low–latency access to individual incoming and outgoing audio sample values. It provides format conversion between the hardware audio formats and your application audio format, doing so by way of an included Format Converter unit. For sample code that shows how to use the Remote I/O unit, see the sample code project aurioTouch.
> The Voice-Processing I/O unit extends the Remote I/O unit by adding acoustic echo cancelation for use in a VoIP or voice-chat application. It also provides automatic gain correction, adjustment of voice-processing quality, and muting.
> The Generic Output unit does not connect to audio hardware but rather provides a mechanism for sending the output of a processing chain to your application. You would typically use the Generic Output unit for offline audio processing.
> The osxaudiosrc plugin is using Remote I/O unit but for it is not suited for VoIP purpose :-( 
> In gstosxcoreaudio.c I see:
> #ifdef HAVE_IOS
> #include "gstosxcoreaudioremoteio.c"
> #else
> #include "gstosxcoreaudiohal.c"
> #endif
> 
> To try to solve the issue I’ve modified this section in gstosxcoreaudioremoteio.c 
> 
> static gboolean
> gst_core_audio_open_impl (GstCoreAudio * core_audio)
> {
>   return gst_core_audio_open_device (core_audio, kAudioUnitSubType_VoiceProcessingIO,
>                                        "VoiceProcessingIO");
>   //return gst_core_audio_open_device (core_audio, kAudioUnitSubType_RemoteIO,
>   //    "RemoteIO");
> }
> Unfortunately it’s not enough because with this patch It is not enough, I’ll work on that and find why it is not working but.. in case you already have experience with  kAudioUnitSubType_VoiceProcessingIO I appreciate your help.
> 
> Thanks
> Elio
> On 05 Jul 2014, at 11:14, Sebastian Dröge <sebastian at centricular.com> wrote:
> 
>> On Sa, 2014-07-05 at 10:43 +0200, Elio Francesconi wrote:
>>> Hi all,
>>> thanks to your supposrt I’ve released a voip application based on GStreamer :-) https://itunes.apple.com/it/app/ipdoor/id881935787?mt=8
>>> Unfortunately our customer are complaining about acoustic echo on iPad devices.
>> 
>> Thanks for letting us know, that's good news :) More GStreamer apps in
>> the app store, or at least more we know about ;)
>> 
>>> I’m using GStreamer version 1.2.4 and I’ve added a schema about my two pipelines, it seems osxaudiosink and osxaudiosrc generate a bad acoustic echo, 
>>> 
>>> Someone has experience on such kind of issues?
>>> [...]
>> 
>> What kind of echo is this? Is the microphone capturing the speaker
>> output again, i.e. are sender and receiver close enough for that to
>> work? Do you also get an echo if you write to a file instead of directly
>> outputting it on the speakers?
>> 
>> If it's the normal echo you get when capturing what your speakers
>> output, and then output that captured audio again... then you need to
>> insert an echo cancellation filter somewhere in your application between
>> the capture and output. Which might be a bit tricky because capture and
>> output is on different devices in your case.
>> 
>> -- 
>> Sebastian Dröge, Centricular Ltd - http://www.centricular.com
>> Expertise, Straight from the Source
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140708/45f4ed57/attachment-0001.html>