vaapih264enc and x264enc usage for encoding MVC / stereo?
Jérôme Carretero
cJ-gst at zougloub.eu
Sat Jan 28 19:11:40 UTC 2017
Hi,
I will comment on the bug, trying to get further into encoding, first:
from your answers, progress was made but there still something that's
not right.
Reference encodings with vaapi and x264 (I am generating a test video
that I assume has stacked side-by-side left-right views in the same
frame buffer):
gst-launch-1.0 --verbose --no-fault \
videotestsrc num-buffers=60 \
! video/x-raw,format=I420,width=2048,height=1024,framerate=30/1 \
! vaapih264enc rate-control=cbr bitrate=2000 \
! h264parse \
! matroskamux streamable=true \
! filesink location=tmp-va.mkv
gst-launch-1.0 --verbose --no-fault \
videotestsrc num-buffers=60 \
! video/x-raw,format=I420,width=2048,height=1024,framerate=30/1 \
! queue max-size-time=10000000000 \
! x264enc bitrate=2000 \
! h264parse \
! matroskamux streamable=true \
! filesink location=tmp-x264.mkv
Trying to use num-views=2 with vaapi:
gst-launch-1.0 --verbose --no-fault \
videotestsrc num-buffers=60 \
! video/x-raw,format=I420,width=2048,height=1024,framerate=30/1,multiview-mode=side-by-side \
! vaapih264enc rate-control=cbr bitrate=2000 num-views=2 \
! h264parse \
! matroskamux streamable=true \
! filesink location=tmp-va-sbs.mkv
mediainfo tmp-va-sbs.mkv says:
MultiView_Count : 2
MultiView_Layout : Right Eye
(not sure why "Right Eye" is said):
During playback, the image has an aspect ratio of 4:1 instead of 2:1:
Input #0, matroska,webm, from 'tmp-va-mvc.mkv':
Metadata:
encoder : GStreamer matroskamux version 1.10.2
creation_time : 2017-01-28T19:01:48.000000Z
Duration: N/A, start: 0.033000, bitrate: N/A
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 2048x1024 [SAR 1:1 DAR 2:1], SAR 2:1 DAR 4:1, 30 fps, 30 tbr, 1k tbn, 60 tbc (default)
Metadata:
title : Video
stereo_mode : left_right
Side data:
stereo3d: side by side
And there are lots of:
Error while decoding frame!
[NULL @ 0x7fa732aa7b60]missing picture in access unit with size 16898
[h264 @ 0x7fa732aa7b60]no frame!
Then I tried using x264 to see what happens:
gst-launch-1.0 --verbose --no-fault \
videotestsrc num-buffers=60 \
! video/x-raw,format=I420,width=2048,height=1024,framerate=30/1,multiview-mode=side-by-side \
! queue max-size-time=10000000000 \
! x264enc bitrate=2000 \
! h264parse \
! matroskamux streamable=true \
! filesink location=tmp-x264-sbs.mkv
Same mediainfo, same unexpected aspect ratio, but no playback warnings.
Best regards,
--
Jérôme
On Fri, 27 Jan 2017 09:39:35 +0100
Víctor M. Jáquez L. <vjaquez at igalia.com> wrote:
> On 01/26/17 at 06:25pm, Sreerenj Balachandran wrote:
> > set "num-views=2" to vaapih264enc
> >
> > On Thu, Jan 26, 2017 at 6:22 PM, Jérôme Carretero
> > <cJ-gst at zougloub.eu> wrote:
> >
> > > Hi,
> > >
> > >
> > > Did not get a reply... is there any documentation available or I
> > > should dig in the code?
>
> Oops! Sorry, this thread slip out of my head.
>
> As Sree already replied, with the property num-views you should set
> MVC encoding.
>
> Could you comment this issue in bug
>
> https://bugzilla.gnome.org/show_bug.cgi?id=757941 ???
>
> To keep it in mind.
>
> Thanks
>
> vmjl
>
> > >
> > >
> > > Thanks,
> > >
> > > --
> > > cJ
> > >
> > >
> > > On Wed, 28 Dec 2016 17:49:37 -0500
> > > Jérôme Carretero <cJ-gst at zougloub.eu> wrote:
> > >
> > > > Hi Víctor,
> > > >
> > > >
> > > > (adding Jan because I saw
> > > > https://gstreamer.freedesktop.org/data/events/gstreamer-
> > > conference/2015/Jan%20Schmidt%20-%20Stereoscopic%203D%20Redux.pdf)
> > > >
> > > >
> > > > Yes, the hardware supports MVC encoding (intel SkyLake).
> > > > Definitely the fact that the downstream caps weren't used wasn't
> > > > helping there.
> > > >
> > > > After some further Googling based on your hint “MVC encoding is
> > > > selected is if the number of views in upstream is equal or
> > > > bigger than 2”, I found in the glstereomix docs the
> > > > "multiview-mode=side-by-side" which is added at the output of
> > > > glstereomix.
> > > >
> > > >
> > > > Because I already have mixed video in one of my cases
> > > > (top-bottom or left-right), so I tried:
> > > >
> > > > ! ... \
> > > > ! video/x-raw,format=I420,multiview-mode=side-by-side \
> > > > ! vaapih264enc \
> > > > ! h264parse ! ...
> > > >
> > > > But to no avail.
> > > >
> > > >
> > > > Is the StereoHigh encoder expecting these side-by-side buffers,
> > > > and I'm missing "metadata", or does it prefer to use separate
> > > > buffers?
> > > >
> > > >
> > > > To "reproduce" a similar situation:
> > > >
> > > > # Get a random SBS video on youtube, I believe it's clearly
> > > > fair use
> > > >
> > > > youtube-dl EgD5_7u2NvQ
> > > > mv *EgD5_7u2NvQ*.mp4 sbs.mp4
> > > >
> > > > # Attempt to recompress using StereoHigh
> > > >
> > > > gst-launch-1.0 --verbose --no-fault \
> > > > filesrc location=sbs.mp4 \
> > > > ! qtdemux name=demux \
> > > > demux.audio_0 ! queue ! fakesink \
> > > > demux.video_0 \
> > > > ! queue name=q_dec\
> > > > ! h264parse \
> > > > ! avdec_h264 \
> > > > ! vaapipostproc \
> > > > ! queue name=q_enc \
> > > > ! video/x-raw,format=I420,multiview-mode=side-by-side \
> > > > ! vaapih264enc \
> > > > ! h264parse \
> > > > ! mp4mux fragment-duration=1000 \
> > > > ! filesink location=test-stereohigh.mp4
> > > >
> > > > GST_DEBUG=GST_CAPS:5 gst-launch-1.0 \
> > > > filesrc location=test-stereohigh.mp4 ! qtdemux ! h264parse !
> > > > fakesink \
> > > > 2>&1 | grep stream-format | head -n 1
> > > >
> > > > # video/x-h264, parsed=(boolean)true,
> > > > stream-format=(string){ avc, avc3, # byte-stream },
> > > > alignment=(string){ au, nal }
> > > >
> > > >
> > > > Regards,
> > > >
> > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > >
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list