[gst-devel] Audio visualizations
landimatte at gmail.com
Fri Aug 6 13:48:12 CEST 2010
On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <ensonic at hora-obscura.de> wrote:
> Am 27.07.2010 18:53, schrieb Matteo Landi:
>> Hi guys,
>> these days I was wondering how to use gstreamer in order to create
>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>> The application consist of a window with a drawing area used to
>> display the visualizations.
>> Regarding the pipeline, I thought about something like
>> filesrc !
>> decodebin !
>> audioconvert !
>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>> tee name=t !
>> queue !
>> fakesink name=fakesink signal-handoffs=true t. !
>> queue !
>> with the handoff callback linked to a method which spawn a new thread,
>> process the signal, and queue a redraw of the window.
> Have a look at gst-plugins-good/test/examples/spectrum/
Thank you, at first I haven't noticed the demo used to display bars
values read from the spectrum built in plugin. I was trying to
re-implement my own just for the curiosity-sake, nothing else :P
> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
> a gst element and post the data via GstBus to the main thread and draw there.
Indeed, I suppose I was not been too clear while explaining my approach:
1 the fakesink queue a callback linked to the event (this is equal to
listening for messages on the bus): the only difference is that I need
more time for audio processing (fft) inside that callback
2 invoke a redraw method using gobject.idle_add with the fft data
processed at the previous step
3 draw the content on a backup cairo surface and invalidate the window
4 automatically redraw the window (paint the surface) using the
generated expose event.
The main difference I see with the code you linked , is that I
don't care at all about all those clocks operations (are they needed?)
I talked about threads, because I was thinking about spawning a new
thread for the audio process:
1 fakesink callback which spawns a new thread
2 threaded audio processing
3 gobject.idleadd ..
There last one obscure thing: why the demo example does not initialize
threads for gobject? I thought it was necessary while working
simultaneously with gtk and gst..
>> I got few problems with the implementation:
>> 1 - I often get core dumped, and I cannot understand why: all the
>> drawing actions are done inside the main thread, so it should be ok
>> 2 - Synchronization between audio and video: the two sources are not
>> synchronized.. maybe is it something related to the queues elements?
>> Have you got any advices regarding the chosen implementation, and how
>> to solve the listed problems?
>> Thanks in advance for the attention.
More information about the gstreamer-devel