[Bug 660770] videodecoder, videoencoder: support "partial" frames / slices for raw video

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=660770

--- Comment #12 from Edward Hervey <bilboed at bilboed.com> ---
(In reply to Guillaume Desmottes from comment #11)
> So to summarize we currently have 3 suggested ways to implement
> subframes/slices:
> 
> a) Adding a meta and/or caps feature, have the GstMemory being progressively
> filled (Edward's suggestion) and map() blocking until the operation is
> complete.
> 
> Pro:
> - Shouldn't break compat with unmodified elements
> 
> Cons:
> - Kind of hacky

  Hacky in what sense ?

> - Mapping is no longer deterministic

  It never was deterministic ? There's no guarantee you'll get the map
immediately (because it requires doing hardware fetches for example) or at all
for that matter anywhere in GStreamer.

  Nothing prevents you from having an additional mapping API (like is done with
other meta systems). And with the proper capsfeature one can avoid providing
such buffer types.

(In reply to Nicolas Dufresne (ndufresne) from comment #10)
> (In reply to Edward Hervey from comment #7)
> > Great that people are interested in this ! My thoughts:
> > 
> > We might not actually need new caps, but instead a new
> > GstMeta/API/CapsFeature.
> > 
> > You still have regular buffers, but the backing GstMemory is progressively
> > filled (or read).

  Make that : the backing *memories*.

> > 
> > You have a new GstMeta/API to allow progressive reading/writing of the
> > backing memory.
> > 
> > If you don't support the meta, doing a _map() will block until all the
> > content has been written/read by the previous element supporting the stride
> > APi.
> 
> I don't like the idea of hacking GstMemory/GstBuffer map to implement
> implicit fencing. It's just a bad idea, prone to stall, hard to interrupt,
> and complete with multiple GstMemory objects. Mapping a buffer should be
> deterministic. I'd say, if downstream does not support that, reconstruct
> locally and push full frame.

  The point of having explicit map in GStreamer *is* to support fencing and
cache invalidation if needed (in addition to allowing different memory
handlers).

  With the capsfeature, upstream can decide what it wants to do. If all of
downstream supports that feature => yay. If downstream doesn't => reconstruct.

> 
> I don't think this apply to encoded data, which unfixed size. For raw data,
> this model applies to decoders that would allocate one buffer per frame, but
> if we have once buffer per slice, it kind of fall apart. Would be nice to
> see what is being done in the field, e.g. jpeg200, openh264 have that
> notion, ALG OMX etc.
> 
> > 
> > If you do support the meta, you can then do progressive READ/WRITE.
> > 
> > As an element supporting the meta, you can push the buffer downstream as
> > soon as there's at least one "slice" in it and you have properly filled the
> > meta.
> 
> No need to wait really, you could push as soon as the timestamp are known.

  You'd still want to do some minor validation/checks before pushing
downstream. Like whether the data *can* be handled.

> 
> > 
> > The meta could contain information such as:
> > * slice height
> > * slices available (that can be mapped by downstream)
> > * total number of slices
> > * method to do slice-based map (read/write)
> > 
> > for backward compatibility the memory should have fallback map
> > implementations to wait for all slices to be available before returning the
> > full map.
> 
> The use of singular "memory" is bogus, also I'm not a big fan of these
> map/unmap function. The one in GstVideoMeta haven't served very well, most
> of the time, it get implemented to hide a bug in the offsets array.

  There's no obligation to make the extra API a *map* like API. If a
direct-access method can be provided, that can also be an option. The only
required is to have a fallback map-based system.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.