[Bug 693037] ximagesrc: delay image reconstruction when using DAMAGE

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=693037
  GStreamer | gst-plugins-good | git

--- Comment #6 from Edward Hervey <bilboed at bilboed.com> 2013-10-31 07:00:02 UTC ---
(In reply to comment #3)
> This sounds like a good design, however I have a use-case where I'd like to
> convert the frames with damage data into jpegs which are transparent except for
> the changed data. That way I can simply overlay them on top of each other at
> the display end, and never have to reconstitute the images in memory. Although,
> I suppose it would make sense if every Nth frame were a full frame, for ease of
> recovery in case of loss of a frame.

At the top of my head :

Indeed, you should be able to handle both usages (one for plugins who are not
aware or care about damaged region and want whole (reconstructed) frames, and
the other one for plugins who could make a more efficient usage of the damaged
regions).

There are two parts to it:
* The actual data storage (i.e. custom GstMemory)
* The metadata about those damaged regions

In order for this system to work, you'll need a common (shared amongst all
buffers/memory of a stream) structure to contain the last fully-reconstructed
image/GstMemory. This could/should be stored in a GstBufferPool imho.

For the "damaged" GstMemory you want to know (and privately remember):
* Whether it is a whole or to-be-reconstructed image
* If to-be-reconstructed, a reference to the previous GstMemory (which will
chain all the way to the last fully reconstructed image).
* a list of
** GstMemory for this image's partial memory (those are not stored in
GstBuffer, otherwise the default gst_buffer_map would give you garbage).
** the x,y pixel offset of that memory in the whole frame
** the width,height pixel offset of that memory in the whole frame
* Implementations of map/unmap vmethod to reconstruct if needed (i.e. the
fallback solution). Those would then chain-up and reconstruct a full video
image (which further buffers/memory will then use).

That would take care of the "naive" delegated reconstruction (any plugin
downstream wanting to map the memory will end up with a full image).

You'll also need to create a GstMeta to provide information on the "damaged"
content, for the plugins that can handle it
* Glist of damage information
  * x,y position in pixels
  * width, height in pixels
  * GstMemory reference of the individual damaged area

That would allow plugins that know how to deal with damaged images to be able
to get the individual regions.

You would then need to add support to your encoder to create those
"transparent" images you mention.

The x,y,width,height would all be in pixels and you'd have to check the
GstVideoMeta for the actual type and compute the bytes offsets/strides

Hope this helps

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.