vaapih264enc and x264enc usage for encoding MVC / stereo?

[Jérôme Carretero]

Hi Víctor,


(adding Jan because I saw
https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Jan%20Schmidt%20-%20Stereoscopic%203D%20Redux.pdf)


Yes, the hardware supports MVC encoding (intel SkyLake).
Definitely the fact that the downstream caps weren't used wasn't
helping there.

After some further Googling based on your hint “MVC encoding is selected
is if the number of views in upstream is equal or bigger than 2”, I
found in the glstereomix docs the "multiview-mode=side-by-side" which
is added at the output of glstereomix.


Because I already have mixed video in one of my cases (top-bottom or
left-right), so I tried:

  ! ... \
  ! video/x-raw,format=I420,multiview-mode=side-by-side \
  ! vaapih264enc \
  ! h264parse ! ...

But to no avail.


Is the StereoHigh encoder expecting these side-by-side buffers, and I'm
missing "metadata", or does it prefer to use separate buffers?


To "reproduce" a similar situation:

# Get a random SBS video on youtube, I believe it's clearly fair use

youtube-dl EgD5_7u2NvQ 
mv *EgD5_7u2NvQ*.mp4 sbs.mp4

# Attempt to recompress using StereoHigh

gst-launch-1.0 --verbose --no-fault \
 filesrc location=sbs.mp4 \
 ! qtdemux name=demux \
 demux.audio_0 ! queue ! fakesink \
 demux.video_0 \
 ! queue name=q_dec\
  ! h264parse \
  ! avdec_h264 \
  ! vaapipostproc \
 ! queue name=q_enc \
  ! video/x-raw,format=I420,multiview-mode=side-by-side \
  ! vaapih264enc \
  ! h264parse \
  ! mp4mux fragment-duration=1000 \
  ! filesink location=test-stereohigh.mp4

GST_DEBUG=GST_CAPS:5 gst-launch-1.0 \
 filesrc location=test-stereohigh.mp4 ! qtdemux ! h264parse ! fakesink \
 2>&1 | grep stream-format | head -n 1

# video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, avc3,
# byte-stream }, alignment=(string){ au, nal }


Regards,

-- 
Jérôme

On Wed, 28 Dec 2016 19:38:22 +0100 Víctor M. Jáquez L.
<vjaquez at igalia.com> wrote:

> Hi,
> 
> First thing to check is does your hardware support MVC?
> 
> You can know that by running `vainfo`
> 
> And you should see
> 
>       VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
>       VAProfileH264StereoHigh         : VAEntrypointEncSlice
> 
> 
> Second thing is that setting the profile via capsfilters is not
> supported yet by the vaapi encoders:
> https://bugzilla.gnome.org/show_bug.cgi?id=757941
> 
> Finally, I haven't tested MVC encoding personally, but grepping the
> source code the way how MVC encoding is selected is if the number of
> views in upstream is equal or bigger than 2.
> 
> 
> vmjl
> 
> On 12/28/16 at 12:32pm, Jérôme Carretero wrote:
> > Hi,
> > 
> > 
> > 
> > I'm running out of google-fu to find documentation on how it's
> > possible to try and use h264 encoding with MVC profiles.
> > 
> > 
> > I am under the impression that the following should work to encode
> > side-by-side stereoscopic video:
> > 
> > ... ! vaapih264enc ! video/x-h264,profile=stereo-high !
> > h264parse ! ...
> > 
> > But the resulting videos don't seem to be in the right profile.
> > 
> > 
> > Then, there is MVC, and I really had trouble finding *anything* on
> > that.
> > 
> > 
> > 
> > Best regards,