Decklink multi-channel audio source

[John P Poet]

On Wed, Aug 31, 2016 at 1:00 AM Sebastian Dröge <sebastian at centricular.com>
wrote:

> On Tue, 2016-08-30 at 19:41 +0000, John P Poet wrote:
> > On Tue, Aug 30, 2016 at 12:42 AM Sebastian Dröge <
> sebastian at centricular.com> wrote:
> > > On Mon, 2016-08-29 at 21:03 +0000, John P Poet wrote:
> > > > I noticed that support for 8 and 16 channel audio was added to
> > > > decklinkaudiosink.  Are there any plans to support to
> > > > decklinkaudiosrc as well?
> > > >
> > > > I came across
> https://bugzilla.gnome.org/attachment.cgi?id=274910&act
> > > > ion=diff , but that is quite old.  I have not even tried to apply it.
> > >
> > > It does not apply anymore and needs a major update.
> > >
> > > Adding support for 8 and 16 channels is relatively simple and can be
> > > done similar to what that patch does, however decklink does not provide
> > > any information about the location/layout of channels and as such we
> > > would have to handle it as unpositioned.
> > >
> > > Do you want to update the patch?
> >
> > I am willing to try.  I have hacked on other peoples code a little bit,
> when necessary, but do not have a high degree of comfort when working on
> GStreamer internals.
> >
> > Is there some other src module I can use as an example?  Searching for
> similar code, I see:
> >
> > gst-plugins-bad/ext/flite/gstflitetestsrc.c
> > gst-plugins-base/gst/audioconvert/gstaudioconvert.c
> >
> > Based on those two, I am guessing I need to create a "fixate" function.
> I need to understand how the caps are managed...
>
> You would need to:
> 1) Add 8 / 16 channels to the template caps like in the audio
> 2) Handle those in gst_decklink_audio_src_set_caps() (EnableAudioInput)
> 3) Check what the hardware actually supports and do something like
>    in gst_decklink_audio_sink_get_caps()
> 4) Implement fixate to select a meaningful number of channels if
>    you still get multiple in there. 2 seems like the best choice
>    as we know the channel layout for that
>
> In general for 8 / 16 channels, the channel-mask in the caps would be
> 0, which means unpositioned.
>
> --
> Sebastian Dröge, Centricular Ltd · http://www.centricular.com
>

Hi Sebastian,

I am attaching my attempt at this.  Unfortunately it does not quite work.
I cannot figure out how to get 8 channels of audio encoded into ac3.  With
this pipeline:

```
gst-launch-1.0 decklinkvideosrc device-number=${DEVICE} mode=0 ! queue !
autovideoconvert ! deinterlace mode=2 ! queue ! avenc_mpeg2video
bitrate=800000 ! mpegvideoparse ! mpegtsmux name=mux ! filesink
location=video.ts decklinkaudiosrc device-number=${DEVICE} audio-channels=8
! audioconvert ! avenc_ac3 bitrate=640000 ! ac3parse ! queue ! mux.
```

The resulting file has:
```
Audio
ID                                       : 66 (0x42)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : CM (complete main)
Format settings, Endianness              : Big
Codec ID                                 : 129
Duration                                 : 3s 485ms
Bit rate mode                            : Constant
Bit rate                                 : 640 Kbps
Channel(s)                               : 1 channel
Channel positions                        : Front: C
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Compression mode                         : Lossy
Stream size                              : 272 KiB (15%)
Language                                 : English
```

I have read that the audio from the Decklink is single-track, mult-channel
PCM.  I thought maybe I needed to deinterlace it for the ac3 encoder to
process it correctly, so I tried:
```
 gst-launch-1.0 decklinkvideosrc device-number=${DEVICE} mode=0 ! queue !
autovideoconvert ! deinterlace mode=2 ! queue ! avenc_mpeg2video
bitrate=800000 ! mpegvideoparse ! mpegtsmux name=mux ! filesink
location=video.ts decklinkaudiosrc device-number=${DEVICE} audio-channels=8
! audioconvert ! "audio/x-raw,channels=8" ! deinterleave ! audioconvert !
avenc_ac3 bitrate=640000 ! ac3parse ! queue ! mux.
```
But either that pipeline is incorrect for that purpose, or it just doesn't
help.

I built and ran
https://github.com/lu-zero/bmdtools/blob/master/bmdcapture.cpp ,  and it
produces 8 channel PCM audio.  As far as I can tell, that code is not doing
any conversion, it just is writing the PCM audio straight from the Decklink
card/driver.

I tried coming up with a pipeline which does not convert the audio, but was
unsuccessful.

I also found the patch that added 8/16 channel support to ffmpeg:
https://ffmpeg.org/pipermail/ffmpeg-devel/2015-January/167444.html

but it does not really shed any light on the situation.

Note: If I set the channel mask to zero, then those pipelines don't run at
all.

Any help would be appreciated.

Thanks,

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160907/6c05b49f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: decklink-multichannel.patch
Type: text/x-patch
Size: 10042 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160907/6c05b49f/attachment-0001.bin>