[Bug 660770] videodecoder, videoencoder: support "partial" frames / slices for raw video

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=660770

Nicolas Dufresne (stormer) <nicolas at ndufresne.ca> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nicolas at ndufresne.ca

--- Comment #10 from Nicolas Dufresne (stormer) <nicolas at ndufresne.ca> ---
(In reply to Edward Hervey from comment #7)
> Great that people are interested in this ! My thoughts:
> 
> We might not actually need new caps, but instead a new
> GstMeta/API/CapsFeature.
> 
> You still have regular buffers, but the backing GstMemory is progressively
> filled (or read).
> 
> You have a new GstMeta/API to allow progressive reading/writing of the
> backing memory.
> 
> If you don't support the meta, doing a _map() will block until all the
> content has been written/read by the previous element supporting the stride
> APi.

I don't like the idea of hacking GstMemory/GstBuffer map to implement implicit
fencing. It's just a bad idea, prone to stall, hard to interrupt, and complete
with multiple GstMemory objects. Mapping a buffer should be deterministic. I'd
say, if downstream does not support that, reconstruct locally and push full
frame.

I don't think this apply to encoded data, which unfixed size. For raw data,
this model applies to decoders that would allocate one buffer per frame, but if
we have once buffer per slice, it kind of fall apart. Would be nice to see what
is being done in the field, e.g. jpeg200, openh264 have that notion, ALG OMX
etc.

> 
> If you do support the meta, you can then do progressive READ/WRITE.
> 
> As an element supporting the meta, you can push the buffer downstream as
> soon as there's at least one "slice" in it and you have properly filled the
> meta.

No need to wait really, you could push as soon as the timestamp are known.

> 
> The meta could contain information such as:
> * slice height
> * slices available (that can be mapped by downstream)
> * total number of slices
> * method to do slice-based map (read/write)
> 
> for backward compatibility the memory should have fallback map
> implementations to wait for all slices to be available before returning the
> full map.

The use of singular "memory" is bogus, also I'm not a big fan of these
map/unmap function. The one in GstVideoMeta haven't served very well, most of
the time, it get implemented to hide a bug in the offsets array.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.