[Bug 796519] Add AV1 codec parser

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=796519

--- Comment #17 from XuGuangxin <guangxin.xu at intel.com> ---
(In reply to sreerenj from comment #16)
> (In reply to Georg Ottinger from comment #14)
> > > Please Correct me if I'm wrong,
> > > IIUC AV1 is it always packetized (one full frame in a buffer with all
> > > necessary OBUs in it). right? If so, why can't we do it similar to vp9? Only
> > > one API which accepts the input raw data and parse+fill all the necessary
> > > structures (sequece, frame etc) if the corresponding obus are present?
> > > 
> > > Or do you think it is possible to have the container formats to send only
> > > OBUs to the decoder instead of the whole frame (similar to H264 and HEVC)?
> > 
> > hmm, I am not 100% sure and I must admit that I don't fully understand the
> > packetized concept - but I can sum up what I think AV1 does:
> > 
> > AV1 is structured in 
> > 1.) temporal units
> > 2.) frame units
> > 3.) OBUs
> > 
> > A temporal unit for example might look like this:
> > 
> > *Temporal Delimiter OBU
> > *Sequence Header OBU
> > *Frame OBU
> > 
> > it is marked by a "Temporal Delimiter OBU" and lasts before the next
> > "Temporal Delimiter OBU" starts.
> > 
> > 
> > but it can also look like this (for subsequent frames):
> > 
> > * Temporal Delimiter OBU
> > * Frame OBU
> > 
> > Note: all subsequent temporal units need the original Sequence Header
> > information in order to be decodeable.
> 
> I think we are dealing the same thing in vp9 parser by 1) keeping a void *
> structure(GstVp9ParserPrivate *) in public header 2) few attributes kept
> inside public header.
> 
> Is it different from what you are talking about?
Agree with Sree, maybe we need to hide the detail in private cpp. If we expose
too many things to the user, they need to know too many details about the vc1
spec.

> 
> > 
> > 
> > A Frame OBU is basically the SUM of a Frame Header OBU and a Tile Group OBU
> > (minus one OBU header) and it is on its own a "frame unit"
> > 
> > But a frame unit might also look like this:
> > * Frame Header OBU
> > * 1st Tile Group OBU
> > * ...
> > * Xth Tile Group OBU
> > 
> > -> so there are a number of ways how temporal units and/or frame units might
> > be structured.
> > 
> > It might also be possible that within a temporal unit multiple frame units
> > reside.
> 
> 
> It looks very similar to Superframes in vp9.
> We don't deal superframes in vp9codecparser, but gstreamer-vaapi handle
> this:https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst-libs/
> gst/vaapi/gstvaapidecoder_vp9.c#n540
> 
> I don't remember why we did it like that :(, may be for simplicity..
> 
> 
> > 
> > Annex B for example additionally stores the information how much bytes the
> > actual temporal unit and also the frame unit(s) occupy. (in this case the
> > size of the "packet" is known) - but this is only for Annex B - in low
> > overhead bitstream format (the standard) this is not the case.
> 
> So I guess the container formats (eg:webm) only will communicate the size of
> OBUs, won't allow sending sequence header (or any other obu) separately.
> Also, I don't think we have any example cases to look for this at the moment.
> 
> 
> > 
> > For example the output of dump_obu:
> > 
> > -----
> > $ ./dump_obu ~/SourceCode/aom_testdata/av1-1-b8-01-size-16x16.ivf
> > Temporal unit 0
> >   OBU type:        OBU_TEMPORAL_DELIMITER
> >       extension:   no
> >       length:      2
> >   OBU type:        OBU_SEQUENCE_HEADER
> >       extension:   no
> >       length:      12
> >   OBU type:        OBU_FRAME
> >       extension:   no
> >       length:      169
> >   TU size: 183
> >   OBU overhead:    3
> > Temporal unit 1
> >   OBU type:        OBU_TEMPORAL_DELIMITER
> >       extension:   no
> >       length:      2
> >   OBU type:        OBU_FRAME
> >       extension:   no
> >       length:      77
> >   TU size: 79
> >   OBU overhead:    2
> > File total OBU overhead: 5

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.