[gstreamer-bugs] [Bug 587570] New: Use libavcodec codec's own downmixing

GStreamer (bugzilla.gnome.org) bugzilla-daemon at bugzilla.gnome.org
Wed Jul 1 20:39:05 PDT 2009 

If you have any questions why you received this email, please see the text at
the end of this email. Replies to this email are NOT read, please see the text
at the end of this email. You can add comments to this bug at:
  http://bugzilla.gnome.org/show_bug.cgi?id=587570

  GStreamer | gst-ffmpeg | Ver: git
           Summary: Use libavcodec codec's own downmixing
           Product: GStreamer
           Version: git
          Platform: Other
        OS/Version: All
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: Normal
         Component: gst-ffmpeg
        AssignedTo: gstreamer-bugs at lists.sourceforge.net
        ReportedBy: lrn1986 at gmail.com
         QAContact: gstreamer-bugs at lists.sourceforge.net
     GNOME version: Unspecified
   GNOME milestone: Unspecified


When i proposed to use ffdec_ac3 instead of a52dec as a default ac3 decoder
(ffdec_ac3 is licensed under LGPL and is being maintained by FFmpeg developers,
while liba52 is in -ugly and looks unmaintained), __tim expressed some concerns
about libavcodec's ability to decode and output multichannel audio and to do
correct downstream downmixing negotiation (which supposedly means that
libavocodec should heed the channel count requirements of downstream elements
and use internal codec-specific downmixers to reduce the channel count instead
of relying on audioconvert to do so).

Multichannel output in libavcodec is fairly stable (at least for ac3 decoder),
and it should be, because 5.1 AC3s are very widespread these days.

Some of the libavcodec's codecs (ac3, mlp, dca and faad) are capable of
performing internal downmixing. OK, we aren't very interested in dca (got
dca-based decoder in -bad, with downmixing) and faad (same here, although
libgstfaad does not use downmixing, which is available via
faadcfg->downMatrix), but mlp and ac3 are desirable (since there is no mlp
decoders in GStreamer and since ffdec_ac3 is LGPL), and with downmixing too!

So, i've made a patch that adds downmixing support to GstFFmpegDec. What it
does:
* Exposes request_channels AVCodecContext as "request-channels" property of
ffdec_* elements
* When request_channels is not set, ffmpegdec sets it up automatically to match
the caps of downstream elements
The latter part is a bit shaky, but i hope that it works at least as good as
the similar code in a52dec. Even if it doesn't, it's a good start.

To check the results of my changes, i decided to run a couple of tests. I had
to use vorbis to encode various ac3 decoders' output, since i have to real
multichannel hardware. Vorbis appears to be the only encoder capable of
encoding 6-channel audio, except flac, which encodes it too, but with highly
reduced volume level (dunno why it happens and i had no desire to find that
out). I've performed tests only on combinations of 2 and 6 channel audio, for
simplicity. I've used Audacity to open the resulting ogg-vorbis files and see
the contents.

The test subject was this file - http://www.tfm.ro/ac3/download/test_ac3.rar
(i've used ffmpeg -i "Test AC3 v2.0.avi" -acodec copy surround.ac3 to extract
raw .ac3 audio stream)

The pipelines i've used:

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 !
audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink
location=surround_ffdec_ac3_org_e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 !
audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink
location=surround_ffdec_ac3_org_e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 !
audioconvert ! vorbisenc ! oggmux ! filesink
location=surround_ffdec_ac3_org_default.ogg

Replaced libgstffmpeg with modified version

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3
request-channels=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc !
oggmux ! filesink location=surround_ffdec_ac3_mod_r2e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3
request-channels=6 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc !
oggmux ! filesink location=surround_ffdec_ac3_mod_r6e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3
request-channels=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc !
oggmux ! filesink location=surround_ffdec_ac3_mod_r2e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3
request-channels=6 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc !
oggmux ! filesink location=surround_ffdec_ac3_mod_r6e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 !
audioconvert ! vorbisenc ! oggmux ! filesink
location=surround_ffdec_ac3_mod_default.ogg

For reference, the output from a52dec

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7
lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r6e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2
lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r2e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7
lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r6e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2
lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r2e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7
lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r6e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2
lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r2e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7
lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r6e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2
lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux !
filesink location=surround_a52dec_r2e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 !
audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_r6e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 !
audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_r2e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 !
audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_r6e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 !
audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_r2e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=true !
audioconvert ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_default_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=false !
audioconvert ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_default_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec !
audioconvert ! vorbisenc ! oggmux ! filesink
location=surround_a52dec_default.ogg

Conclusions:
* Despite what g_object_class_install_property (G_OBJECT_CLASS (klass),
ARG_LFE, g_param_spec_boolean ("lfe", "LFE", "LFE", TRUE, G_PARAM_READWRITE));
says, LFE is disabled by default (or maybe a52dec disables it by itself in such
circumstances)
* There is no lfe on/off switch in ffdec_ac3 (lfe is always on)

* Unpatched (request_channel-unaware) ffdec_ac3 outputs 6-channel audio by
default (no downstream downmixing negotiation)

* When lfe is off, a52dec outputs mono audio without lfe by default (negotiates
1 channel?)
* Patched ffdec_ac3 outputs mono audio without lfe by default, just like a52dec
with lfe=false (negotiates 1 channel?)

* When lfe is forced on, a52dec outputs 3-channel audio with lfe by default
(negotiates 3 channels?)
* When 3 channels are requested, patched ffdec_ac3 outputs 3-channel audio with
lfe (although the mixing strategy is different from default a52dec with
lfe=false)

* When lfe is forced on and 2 channels are requested, a52dec outputs stereo
with lfe
* When 2 channels are requested, patched ffdec_ac3 outputs stereo without lfe

* When lfe is forced on and 6 channels are requested, a52dec outputs 6ch audio
with lfe
* When 6 channels are requested, patched ffdec_ac3 outputs 6ch audio with lfe

* When 2 or 3 channels are forced downstream, patched ffdec_ac3 behaves just as
if 2 or 3 channels were explicitly requested from it (which means that
negotiation works as intended)

* It seems that ffmpeg is being "smart" about lfe, discarding it when
low-channel-count output is requested/negotiated, and preserving it when
high-channel-cout output is requested/negotiated

surround.ac3 and the resulting ogg-vorbis files are available here -
http://lrn.no-ip.info/other/gstreamer/surround_ffdec_ac3_vs_a52dec.7z


-- 
See http://bugzilla.gnome.org/page.cgi?id=email.html for more info about why you received
this email, why you can't respond via email, how to stop receiving
emails (or reduce the number you receive), and how to contact someone
if you are having problems with the system.

You can add comments to this bug at http://bugzilla.gnome.org/show_bug.cgi?id=587570.

More information about the Gstreamer-bugs mailing list