[pulseaudio-discuss] Pulseaudio on the ESP32 ?

[JF]

Hello Denis,

I believe there is a misunderstanding, I do not intend to run PA on the
ESP32.

I want to build a wifi headphone with microphone that connects with PA
native or simple protocol.  (module-native-protocol-tcp)

The chip has two stereo I2S busses (the variant I have called ESP32-A1S has
audio ADC/DAC on board)

So for the microphone I would sample at 44khz 16 bit one channel, take the
incoming PCM, compress using esp-libopus into 15 millisecond frames at
256kbit/s complexity 1 and send that via PA native tcp protocol.

It is unclear if PA can receive opus compressed audio from the network. It
is not listed in
https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/SupportedAudioFormats/
If not possible, then whatever the next best thing is to keep frame size
low (and make wifi jitter irrelevant, final design will probably use
802.11ax and ideal RF environment conditions)

For the speakers, I hope to tell PA to start sending data to my device over
the network, again in whatever the best low latency lossy compression is
available.

The hope is not to exceed 100ms total latency including all the buffers.

The ESP32 barely has enough cpu power to do all this, so Plan B is to use a
Raspberry Pi Pico (RP 2040 chip) for the sole purpose of compression of the
microphone audio.
Or to run two ESP32, one just for the microphone and one just for the
speakers.

Regarding opus support in PA  (It is unclear if it is supported, supported
through gstreamer or not supported at all, obviously PCM is highly
inefficient over network and probably unusable over wifi without ginormous
buffers)
https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/483





On Fri, Sep 3, 2021 at 6:04 PM Denis Shulyaka <shulyaka at gmail.com> wrote:

> Hi JF,
>
> PA relies on ALSA drivers for the sound cards, including I²S, so unless
> you have a Linux port on this chip, there's a lot of work. Also, active
> noise cancelation is easier said than done.
>
> Best regards,
> Denis Shulyaka
>
> сб, 4 сент. 2021 г., 00:03 JF <pulseaudio at domn.net>:
>
>> Hello,
>>
>> I would like to know if anyone has ever tried to interface pulseaudio
>> with an ESP32 chip ?
>>
>> This extremely popular 3$ chip has ethernet, wifi 802.11n and bluetooth
>> 4.2+BLE capabilities as well as a pair DMA capable I²S bus capable of
>> operation is both master or slave modes.
>>
>> This is on of the most popular chip currently in use by Arduino IDE
>> users, making it extremely accessible for hobbyist use.
>>
>> Infomation regarding the I2s programming interface
>>
>> https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/i2s.html
>>
>> More information about this chip
>> https://en.wikipedia.org/wiki/ESP32
>>
>> A variant of this chip called the ESP32-A1S, comes with on board audios
>> ADC/DACs ?
>> Although the details are not clear about this one for me. It is a
>> audio-centric variant.
>> https://docs.ai-thinker.com/en/esp32-a1s
>> https://github.com/donny681/esp-adf
>>
>> I've been a subscribe to this mailing list for a while but I have not yet
>> spotted discussion of this chip.  I am under the impression that this chip
>> could be the ideal hobbyist pulseaudio platform.
>>
>> (My intention is to create 3d printed wifi gaming headest with DSP
>> capability and active noise cancellation. I know the doc says wifi's
>> not good enough for audio, but I want to try it out for myself, especially
>> with the latest 802.11ax standard)
>>
>> Thanks !
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/pulseaudio-discuss/attachments/20210907/1c9c3fc8/attachment.htm>