Video with alpha to file

[amindfv at mailbox.org]

On Thu, Mar 09, 2023 at 06:28:19PM -0500, Nicolas Dufresne wrote:
> Hi,
> 
> Le jeu. 9 mars 2023, 09 h 00, amindfv--- via gstreamer-devel <
> gstreamer-devel at lists.freedesktop.org> a écrit :
> 
> > I've got an A420_10LE video stream I'd like to write to a file, preserving
> > its alpha channel.
> >
> > I've found that I can use ProRes, with a pipeline like:
> >
> >     ... ! video/x-raw,format=A420_10LE ! videoconvert !
> > video/x-raw,format=A444_10LE ! avenc_prores ! qtmux ! filesink
> >
> > This works but it's slow and results in massive files (almost 5GB for 30
> > seconds of 1080p video).
> >
> > Are there faster+smaller options (encoders and maybe muxers) for writing
> > to files with alpha? And in the spirit of learning to fish, how could I
> > have found them (e.g. with `gst-inspect`)?
> >

Thanks for the advice. A few questions:

> For higher compression, I'd opt for lossy compression. At 1080p, VP9, H.264
> would be good option. VP9 + webm (matroska) seems the shortest path.
> 
> I'd basically wrap 2 vp9enc into a bin.

Just out of curiosity, is a bin necessary or just for convenience?

> Adding a simple element alphasplit,
> to seperate the stream in two, and finally having a vp9alphacombine that
> takes two vp9 streams and attach the alpha using the existing meta into the
> color stream. The final step would to to add matroskamux support.

`vp9alphacombine` doesn't already exist, right? Also, what is the advantage to encoding as vp9 then combining alphas, vs combining alphas as "video/x-raw" then encoding as vp9? (Of course, I notice `vp9enc` doesn't currently support any pixel formats with alpha).

As to `matroskamux`, would it be getting a "video/x-vp9" stream from the `vp9alphacombine`? If so, is it necessary to add support for vp9 with an alpha channel to matroskamux? Or would matroskamux simply mux without "caring" about the internal vp9 stream?

> But if implementing this is not for you, FFMPEG fully support this, you can
> add FFMPEG element, or wrap it with appsrc/sink.

Are there any links/information about an ffmpeg element? Is this something that's been built before?

Also if going the appsrc/sink route, is the idea to convert GstBuffers to AVFrames and use the ffmpeg/libav libraries?

Thanks,
Tom