[Bug 796519] Add AV1 codec parser

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=796519

--- Comment #16 from sreerenj <bsreerenj at gmail.com> ---
(In reply to Georg Ottinger from comment #14)
> > Please Correct me if I'm wrong,
> > IIUC AV1 is it always packetized (one full frame in a buffer with all
> > necessary OBUs in it). right? If so, why can't we do it similar to vp9? Only
> > one API which accepts the input raw data and parse+fill all the necessary
> > structures (sequece, frame etc) if the corresponding obus are present?
> > 
> > Or do you think it is possible to have the container formats to send only
> > OBUs to the decoder instead of the whole frame (similar to H264 and HEVC)?
> 
> hmm, I am not 100% sure and I must admit that I don't fully understand the
> packetized concept - but I can sum up what I think AV1 does:
> 
> AV1 is structured in 
> 1.) temporal units
> 2.) frame units
> 3.) OBUs
> 
> A temporal unit for example might look like this:
> 
> *Temporal Delimiter OBU
> *Sequence Header OBU
> *Frame OBU
> 
> it is marked by a "Temporal Delimiter OBU" and lasts before the next
> "Temporal Delimiter OBU" starts.
> 
> 
> but it can also look like this (for subsequent frames):
> 
> * Temporal Delimiter OBU
> * Frame OBU
> 
> Note: all subsequent temporal units need the original Sequence Header
> information in order to be decodeable.

I think we are dealing the same thing in vp9 parser by 1) keeping a void *
structure(GstVp9ParserPrivate *) in public header 2) few attributes kept inside
public header.

Is it different from what you are talking about?

> 
> 
> A Frame OBU is basically the SUM of a Frame Header OBU and a Tile Group OBU
> (minus one OBU header) and it is on its own a "frame unit"
> 
> But a frame unit might also look like this:
> * Frame Header OBU
> * 1st Tile Group OBU
> * ...
> * Xth Tile Group OBU
> 
> -> so there are a number of ways how temporal units and/or frame units might
> be structured.
> 
> It might also be possible that within a temporal unit multiple frame units
> reside.


It looks very similar to Superframes in vp9.
We don't deal superframes in vp9codecparser, but gstreamer-vaapi handle
this:https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst-libs/gst/vaapi/gstvaapidecoder_vp9.c#n540

I don't remember why we did it like that :(, may be for simplicity..


> 
> Annex B for example additionally stores the information how much bytes the
> actual temporal unit and also the frame unit(s) occupy. (in this case the
> size of the "packet" is known) - but this is only for Annex B - in low
> overhead bitstream format (the standard) this is not the case.

So I guess the container formats (eg:webm) only will communicate the size of
OBUs, won't allow sending sequence header (or any other obu) separately.
Also, I don't think we have any example cases to look for this at the moment.


> 
> For example the output of dump_obu:
> 
> -----
> $ ./dump_obu ~/SourceCode/aom_testdata/av1-1-b8-01-size-16x16.ivf
> Temporal unit 0
>   OBU type:        OBU_TEMPORAL_DELIMITER
>       extension:   no
>       length:      2
>   OBU type:        OBU_SEQUENCE_HEADER
>       extension:   no
>       length:      12
>   OBU type:        OBU_FRAME
>       extension:   no
>       length:      169
>   TU size: 183
>   OBU overhead:    3
> Temporal unit 1
>   OBU type:        OBU_TEMPORAL_DELIMITER
>       extension:   no
>       length:      2
>   OBU type:        OBU_FRAME
>       extension:   no
>       length:      77
>   TU size: 79
>   OBU overhead:    2
> File total OBU overhead: 5

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.