Is videoscale affecting alpha value?

[Peter Maersk-Moller]

Hi Sebastian.

Thanks for answering. Please see replies inline.


On Fri, Dec 20, 2013 at 11:31 AM, Sebastian Dröge <sebastian at centricular.com
> wrote:

> On Fr, 2013-12-20 at 02:08 +0100, Peter Maersk-Moller wrote:
> > Hi
> >
> > I've come across a problem with alpha values where videoscale in some
> cases
> > seems to affect the alpha value in a not so good way for GStreamer 1.2.0
> > (and 1.2.1) in a way that was not the case for version 0.10.
> >
> > When using the following pipeline, the alpha value of each pixels gets
> set
> > to 127, if the content decoded is not 1280x720 and set to 255, if the
> > content is 1280x720:
> >
> > /usr/local/bin/gst-launch-1.0 -v filesrc location=../test/pmm2.mp4 !\
> >     decodebin !\
> >     videoscale !\
> >     videoconvert !\
> >     'video/x-raw, format=(string)BGRA, width=(int)1280, height=(int)720'
> !\
> >     shmsink socket-path=/tmp/feed1-control-pipe shm-size=50000000
> > wait-for-connection=0
>
> What's the video format that is output by decodebin here?
>

Ill post the full verbose information later in this email, but the output
og decodebin is I420 at 640x360 with PAT 1/1 progressive colorimetry bt601
and rate 24/1

/GstPipeline:pipeline0/GstDecodeBin:decoder.GstDecodePad:src_0.GstProxyPad:proxypad2:
caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1.

As a doublecheck I find this to be inline with what videoscale reports for
input except that videoscale reports a framerate of 25/1 .... which is a
bit odd we don't see any complaints about.

/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)25/1


Does it
> contain an alpha channel already, and what's the size of the input
> video?
>

No it does not have an alpha channel when leaving decodebin. How could it?
I haven't heard of an encoded video with an alpha track saved and to the
best of my knowledge, decodebin can not output raw video with an alpha
channel nor does it include videoconvert.

The size of the video is 640x360, but it appear that many, if not all
encoded video (all is not checked) smaller than the required 1280x720 will
trigger the alpha value to be set at 127 instead of 255 by assuming
videoconvert, as videoconvert is adding the alpha channel.



>  Please paste the complete output of gst-launch-1.0 -v in good
> (alpha=255) and bad (alpha=127) cases.
>

Logs will be pasted last in email.



> Can you produce a more simple test case that does not involve shmsink
> and a file, or can you make the file available?
>

I can make the file available, but here is how to produce two samples your
self. You can hit ctrl-c after 10 seconds of movie has been produced.

gst-launch-1.0 -e -v videotestsrc pattern=18 is-live=true ! 'video/x-raw,
width=1280, height=720, format=I420' ! videoconvert ! x264enc !
'video/x-h264, profile=main' ! avimux ! filesink location=video1280x720.mp4

gst-launch-1.0 -e -v videotestsrc pattern=18 is-live=true ! 'video/x-raw,
width=1024, height=576, format=I420' ! videoconvert ! x264enc !
'video/x-h264, profile=main' ! avimux ! filesink location=video1024x576.mp4

Now you can try the following two pipelines:

gst-launch-1.0 -q filesrc location=video1280x720.mp4 ! decodebin
name=decoder ! videoscale ! videoconvert ! 'video/x-raw,
format=(string)BGRA, width=(int)1280, height=(int)720' ! fdsink fd=1
|hexdump -c |less

gst-launch-1.0 -q filesrc location=video1024x576.mp4 ! decodebin
name=decoder ! videoscale ! videoconvert ! 'video/x-raw,
format=(string)BGRA, width=(int)1280, height=(int)720' ! fdsink fd=1
|hexdump -c |less

>From the output you should be able to see that the alpha value is set to
255 and 127 respectively.

Below is my extract from the verbose output of the original pipeline:

/usr/local/bin/gst-launch-1.0 -v filesrc
location=../test/LES_TDS_launch.mp4 do-timestamp=true ! decodebin
name=decoder ! videoconvert ! videorate ! videoscale ! videoconvert !
video/x-raw, format=(string)BGRA, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, width=(int)1280, height=(int)720,
framerate=(fraction)25/1 ! identity silent=true ! queue ! shmsink
socket-path=/tmp/feed1-control-pipe shm-size=77414400 wait-for-connection=0
sync=true decoder. ! queue ! audioconvert ! audioresample ! audio/x-raw,
rate=(int)44100, channels=(int)1 ! fdsink fd=3 sync=true
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstTypeFindElement:typefind.GstPad:src:
caps = video/quicktime, variant=(string)iso
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstTypeFindElement:typefind.GstPad:src:
caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0.GstPad:sink_0:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0.GstPad:src_0:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0.GstPad:src_0:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstH264Parse:h264parse0.GstPad:src:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstAacParse:aacparse0.GstPad:src:
caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, level=(string)2, base-profile=(string)lc,
profile=(string)lc, codec_data=(buffer)12100000000000000000000000000000,
rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstFaad:faad0.GstPad:sink: caps
= audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, level=(string)2, base-profile=(string)lc,
profile=(string)lc, codec_data=(buffer)12100000000000000000000000000000,
rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstAacParse:aacparse0.GstPad:sink:
caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, level=(string)2, base-profile=(string)lc,
profile=(string)lc, codec_data=(buffer)12100000000000000000000000000000,
rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstFaad:faad0.GstPad:src: caps
= audio/x-raw, format=(string)S16LE, layout=(string)interleaved,
rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstCapsFilter:capsfilter2.GstPad:src:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decoder/avdec_h264:avdec_h264-0.GstPad:sink:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstCapsFilter:capsfilter2.GstPad:sink:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstH264Parse:h264parse0.GstPad:sink:
caps = video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)constrained-baseline,
codec_data=(buffer)0142c01effe100186742c01eda0280bfe584000003000400000300c03c58ba8001000468ce3c80,
width=(int)640, height=(int)360, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1
Redistribute latency...
/GstPipeline:pipeline0/GstDecodeBin:decoder/avdec_h264:avdec_h264-0.GstPad:src:
caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decoder/GstMultiQueue:multiqueue0:
max-size-bytes = 2097152
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw,
format=(string)S16LE, layout=(string)interleaved, rate=(int)44100,
channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw,
format=(string)S16LE, layout=(string)interleaved, rate=(int)44100,
channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps =
audio/x-raw, layout=(string)interleaved, rate=(int)44100,
format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:src: caps =
audio/x-raw, layout=(string)interleaved, rate=(int)44100,
format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps =
audio/x-raw, layout=(string)interleaved, rate=(int)44100,
format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstFdSink:fdsink0.GstPad:sink: caps = audio/x-raw,
layout=(string)interleaved, rate=(int)44100, format=(string)S16LE,
channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps =
audio/x-raw, layout=(string)interleaved, rate=(int)44100,
format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:sink: caps =
audio/x-raw, layout=(string)interleaved, rate=(int)44100,
format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps =
audio/x-raw, format=(string)S16LE, layout=(string)interleaved,
rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps =
video/x-raw, format=(string)I420, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:src: caps =
video/x-raw, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1, format=(string)BGRA
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
video/x-raw, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1, format=(string)BGRA
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:src: caps =
video/x-raw, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1, format=(string)BGRA
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw,
width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, framerate=(fraction)25/1,
format=(string)BGRA
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw,
width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, framerate=(fraction)25/1,
format=(string)BGRA
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw,
width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, framerate=(fraction)25/1,
format=(string)BGRA
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:sink: caps =
video/x-raw, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1, format=(string)BGRA
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
video/x-raw, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
framerate=(fraction)25/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstDecodeBin:decoder.GstDecodePad:src_0.GstProxyPad:proxypad2:
caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
colorimetry=(string)bt601, framerate=(fraction)24/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20131220/3646dd09/attachment-0001.html>