Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

[Amit Pandya]

Hi all,

Following pipeline used for 4K H264 encode using qtmux pipeline,

*a)  gst-launch-1.0 videotestsrc num-buffers=100 ! 'video/x-raw,
width=(int)3840, height=(int)2160, format=(string)I420' ! x264enc !
h264parse ! qtmux ! filesink
location=test_x264enc_3840x2160_I420_100frames.mp4 -v -e*
....
/GstPipeline:pipeline0/GstX264Enc:x264enc0.GstPad:src: caps = video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstQTMux:qtmux0.GstQTMuxPad:video_0: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono

With above x264enc 4k encode pipelines colorimetry bt2020 is getting
negotiated & same is used by the qtmux element.

*b) gst-launch-1.0 filesrc
location=test_x264enc_3840x2160_I420_100frames.mp4 ! qtdemux ! h264parse !
'video/x-h264, stream-format=(string)byte-stream, alignment=(string)au' !
fakesink -v -e*
....
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)5.1, profile=(string)high,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
width=(int)3840, height=(int)2160, framerate=(fraction)30/1,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
*colorimetry=(string)2:6:0:7*
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
level=(string)5.1, profile=(string)high, width=(int)3840, height=(int)2160,
framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, *colorimetry=(string)2:6:0:7*,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true

When use above pipelines using same encoded mp4 file from a) then qtdemux
provides colorimetry=2:6:0:7 with output caps.
Now, this colorimetry values are different then the format derived
colorimetry values for bt2020=2:6:11:7.

Issue seems to be arising from the way qtmux generates ext_atom for colr_ext
for colorimetry.
while building colr_extension atom "colorimetry->transfer" enum for
GST_VIDEO_TRANSFER_BT2020_12 is not handled.
(Refer gst-plugins-good-1.14.3: gst/isomp4/gstqtmux.c+5594 &
gst/isomp4/atoms.c+4152 )

With qtdemux implementation also while parsing the track for colr_extension
atom respective case is not handled yielding "colorimetry->transfer" enum
value for GST_VIDEO_TRANSFER_UNKNOWN(0).
(Refer gst-plugins-good-1.14.3: gst/isomp4/qtdemux.c+10168)

Above creates problem with the decode pipeline where decoder have
colorimetry equality check for format derived colorimetry
(bt2020-"2:6:11:7") with caps received colorimetry values "2:6:0:7" in
_set_format()

It appears to be missing functionality with qtmux/qtdemux implementation &
enhancement should be done for handling  bt2020 colorimetry value of
GstVideoTransferFunction.

Let me know in case any relevant inputs.

-Thanks.






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/