Splitting audio pipeline with tee and different formats

[Nicolas Dufresne]

Le dimanche 17 novembre 2019 à 01:20 +0100, Jonas Kvinge a écrit :
> Hi
> 
> Sorry for sending this e-mail for the third time. But dmarc reports
> indicated that previous e-mails was blocked by most servers.
> 
> I'm trying to use a tee to split the audio pipeline so that S16LE can
> be sent to a spectrum analyzer, without changing the audio format
> that is sent to alsasink.
> 
> However, whenever I set "format" to "S16LE" for the fakesink, it also
> affects alsasink. So both queues are affected and set to S16_LE. If I
> remove the format and set only "audio/x-raw" or link the elements
> without the caps, it's unaffected and outputs S24_3LE.
> 
> The file is: FLAC audio bitstream data, 24 bit, stereo, 192 kHz,
> 55598080 samples
> 
> I can output S24_3LE correctly with the following command:
> 
> gst-launch-1.0 uridecodebin
> uri=file:///home/jonas/temp/Southern_Cross.flac ! tee name=t t. !
> queue ! audioconvert ! audio/x-raw,format=S24LE ! alsasink
> device=hw:0,0 t. ! queue ! audioconvert ! audio/x-raw,format=S16LE !
> fakesink

I've given another look, and notice that flacdec does not expose S24LE
as an output format, only the 32bit aligned variant (same bits though)
is available. Consider changing your filter into format=S24_32LE. This
will yield the same quality, but will use slightly more bandwidth.

Someone would have to look into the element (flacdec) code, so see if
the packed S24LE can be supported. Another thing that could be improved
in flacdec is the format order:

  format: { (string)S8, (string)S16LE, (string)S24_32LE, (string)S32LE }

I haven't checked in the code, but if upstream element uses
gst_caps_fixate() without any logic, the end result would prefer 8bit,
followed by 16bit, over 24 and 32bit variants. I think we should flip
these caps order, just in case. That being said, many upstream elements
are smarter then this.

regards,
Nicolas