Gstreamer 1.22 and h265 with alpha

[Nicolas Dufresne]

Hi Bill,

Le mer. 25 janv. 2023, 21 h 45, Bill Hofmann via gstreamer-devel <
gstreamer-devel at lists.freedesktop.org> a écrit :

> So, I've got a 1080p HEVC 8bit/component file encoded with alpha. I've
> verified that, for instance, if I play that mp4 file on
> https://rotato.app/tools/transparent-video, it is in fact valid and has
> alpha (Safari on Mac Ventura), so using Apple's HEVC decoder I expect.
>
> When I play it using gst-play:
>
> GST_PLUGIN_FEATURE_RANK=vah265dec:9999 GST_DEBUG=2,buffer*:1 gst-play-1.0
> --use-playbin3 --videosink="kmssink connector-id=308 plane-id=31
> skip-vsync=true" /media/views/content/views/redmonster.mp4
>
>
> I get a passel of these errors:
>
> 0:00:00.069814340 20898 0x7feab00205e0 WARN             h265decoder
> gsth265decoder.c:733:gst_h265_decoder_preprocess_slice:<vah265dec0> Current
> picture is not finished but slice header has first_slice_segment_in_pic_flag
> 0:00:00.069829360 20898 0x7feab00205e0 WARN            videodecoder
> gstvideodecoder.c:4795:_gst_video_decoder_error:<vah265dec0> error: Failed
> to decode data
>
>
> And no picture, or no useful picture.  When instead I use an HLS segmented
> version of it, I *DO* get a picture, but a bad one - it seems to see the
> mask, but not be able to render the image.
>

H.264 and H.265 alpha have not been implemented yet. Normally it should
have played the colors, but the parsers have issues with the alpha slices.
Unlike alpha/webm, the alpha stream is embedded in the bitstream instead of
the container, thus the parser has to be involved.

I have a sample for h.264 alpha, and it decoded fine the colors, could you
file an issue and attach a sample for the h.265 case. We should first fix
the decoding of the colors, and then adding alpha support seems fun project
too.



> Good news is that a vp9/webm version with alpha *does* play like a charm
> using the same magic gst-play command line.
>
> Any thoughts on diagnosing the issue? Is it just "vah265dec doesn't
> support alpha" or is it something more complex?
>

The VP9 alpha support does not yet work with VA, but the autoplugger does
the right thing (plug software decoders, it works with some V4L2 decoders,
not all of them).

The method of encoding 2 streams and combining them is identical. We simply
need the parsers to extract the second stream and create the meta, and then
bins to create the alpha decoders from 2 decoders. Here's a presentation
about VP9/VP8 support.

https://youtu.be/R0flfhLok94

Nicolas



> -Bill
>
>
> --
> Bill Hofmann
> +1 (510) 387-0952
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230126/5e9794df/attachment.htm>