Video with alpha to file

[Nicolas Dufresne]

Le jeu. 9 mars 2023, 22 h 27, amindfv at mailbox.org <amindfv at mailbox.org> a
écrit :

> On Thu, Mar 09, 2023 at 06:28:19PM -0500, Nicolas Dufresne wrote:
> > Hi,
> >
> > Le jeu. 9 mars 2023, 09 h 00, amindfv--- via gstreamer-devel <
> > gstreamer-devel at lists.freedesktop.org> a écrit :
> >
> > > I've got an A420_10LE video stream I'd like to write to a file,
> preserving
> > > its alpha channel.
> > >
> > > I've found that I can use ProRes, with a pipeline like:
> > >
> > >     ... ! video/x-raw,format=A420_10LE ! videoconvert !
> > > video/x-raw,format=A444_10LE ! avenc_prores ! qtmux ! filesink
> > >
> > > This works but it's slow and results in massive files (almost 5GB for
> 30
> > > seconds of 1080p video).
> > >
> > > Are there faster+smaller options (encoders and maybe muxers) for
> writing
> > > to files with alpha? And in the spirit of learning to fish, how could I
> > > have found them (e.g. with `gst-inspect`)?
> > >
>
> Thanks for the advice. A few questions:
>
> > For higher compression, I'd opt for lossy compression. At 1080p, VP9,
> H.264
> > would be good option. VP9 + webm (matroska) seems the shortest path.
> >
> > I'd basically wrap 2 vp9enc into a bin.
>
> Just out of curiosity, is a bin necessary or just for convenience?
>

It is to reuse existing decoders/encoders without modification. It's not
required, but complexity of doing this in a single encoder/decoder is high.
See next answer for more details.


> > Adding a simple element alphasplit,
> > to seperate the stream in two, and finally having a vp9alphacombine that
> > takes two vp9 streams and attach the alpha using the existing meta into
> the
> > color stream. The final step would to to add matroskamux support.
>
> `vp9alphacombine` doesn't already exist, right? Also, what is the
> advantage to encoding as vp9 then combining alphas, vs combining alphas as
> "video/x-raw" then encoding as vp9? (Of course, I notice `vp9enc` doesn't
> currently support any pixel formats with alpha).
>

Maybe this will be compressedalphacombine or something.

The key aspect is that VP9 does not support alpha at all. Instead, convert
the alpha to I420 and later we ignore the chromacity.

It's the same for h.264 and h.265. Except that you don't need third party
muxer capabilities to embed the alpha. The bitstream has auxiliary stream
support.


> As to `matroskamux`, would it be getting a "video/x-vp9" stream from the
> `vp9alphacombine`? If so, is it necessary to add support for vp9 with an
> alpha channel to matroskamux? Or would matroskamux simply mux without
> "caring" about the internal vp9 stream?
>

For vp9, the muxer needs to know, the encoders don't. For h.264/h.265, this
is something to design, but I like the idea of using unmodified encoders
and create element to add the auxiliary into the bitstream.


> > But if implementing this is not for you, FFMPEG fully support this, you
> can
> > add FFMPEG element, or wrap it with appsrc/sink.
>
> Are there any links/information about an ffmpeg element? Is this something
> that's been built before?
>
> Also if going the appsrc/sink route, is the idea to convert GstBuffers to
> AVFrames and use the ffmpeg/libav libraries?
>

I don't have a very helpful link for you, but you'll find the parameters
one may pass to FFMPEG cmdl, I never used that at the C level. I think
kdenlive uses that too.



> Thanks,
> Tom
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230311/c48ad34c/attachment.htm>